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ABSTRACT 

In compliance with mandates of the Elementary and 
Secondary Education Act, the Arizona Department of Education has 
completed its evaluation of effectiveness for state and locally 
administered programs in the context of Arizona's existing Chapter 2 
goals. The -valuation focused first on state-administered programs 
and then considered locally administered programs of training and 
professional development (T&PD) . Benefits documented by the 
evaluation include, but are not limited to the following: (l) T&PD 
programs are a vital resource for Arizona teachers; (2) instructional 
skills and content knowledge that teachers receive have direct 
impacts on over 270,000 students in the school system; (3) the 66 
districts that allocate Chapter 2 funds for T&PD comprise about 75 
percent of the teachers and s tudents in the state; (4) the s tructure 
of Chapter 2 T&PD programs allows teachers and administrators to 
target their own specific needs; (5) T&PD programs contribute 
positively to overall school improvement efforts; and (6) staff 
development activities under Chapter 2 helped inaugurate the Arizona 
Student Assessment Program. Six appendixes describe the evaluation 
methodology and the advisory committee, and include instruments used 
in the evaluation process. Seven charts, 3 tables, and 11 figures 
present study findings. (SLD) 
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Dear Colleagues and Friends of Education: 

Arizona's Evaluation of Chapter 2 Program Effectiveness highlights the collaborative 
efforts undertaken by the Governor's Advisory Committee, the Arizona Department of 
hducation (ADE) and the state's public school districts in response to Congress 1 
accountability mandates. 

In evaluating Chapter 2-supported Training & Professional Development programs, the 
Division of Research and Development and Chapter 2 Office surveyed many educators. 
Their assessments of quality and appropriateness of training were the foundation upon 
which this document is built. 

I would like to thank the 379 district staff who helped evaluate state-administered 
training under the Arizona Student Assessment Program. Although the ADE has a 
strong interest in findings attributable to these individuals and their ADE counterparts 
who participated in regional workshops, our report more broadly points out the depth, 
quality and impacts of locally administered staff development programs. 

I am particularly grateful to the 975 administrators, teachers and support services staff 
who thoughtfully responded to one or more surveys. Forty-three district coordinators 
provided listings of trainees, distributed questionnaires, answered inquiries about 
programs or procedures and assembled materials. Without their cooperation, a major 
portion of Arizona's evaluation could not have been realized. 



Sincerely, 




C. Diane Bishop 

State Superintendent of Public Instruction 
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CENTER FOR EXCELLENCE IN EDUCATION 
Division of Educational/ Field Services 





FROM: 



Dr. Raymond Ver Velde, Chairman - Governor's Advisory Committee 
The Secretary - United States Department of Education 




TO: 



SUBJECT: The Chapter 2 Evaluation of Program Effectiveness 

Many of us, as members of the Governor's Advisory Committee, witnessed sizable changes 
in Arizona's Chapter 2 Program over the past decade. In 1981, we observed consolidation 
of 29 categorical programs into what was then termed the federal education block grant. 

In 1988, under the auspices of the Hawkins-Stafford Elementary and Secondary School 
Improvement Amendments, Congress joined states in acknowledging the successes being 
portrayed locally in terms of innovation, flexibility and services to private school students. 

Advisory Committee members are acquiring a genuine appreciation for the six targeted 
assistance areas now specified within Public Law 100-297. During their frequent 
examinations of program summaries and statistical data, especially at the LEA level, 
members identify many positive trends. 

• Acquisitions in Arizona are increasingly focused upon outcomes. Districts gauge 
benefit, not by how frequently items are checked out, but in light of student 
performance. Refocused and far from a concept of buying "things," use of local 
Chapter 2 funds for library books and reference materials has diminished 21.8% 
over three years. By contrast, amounts budgeted for classroom computers have 
jumped 47.2%, highlighting the value districts ascribe to instructional technology. 

• The Effective Schools Program came under scrutiny last year when the Advisory 
Committee noted that LEAs had not selected this area for funding in the 1989/90 
school year. Members authorized a competitive process for fostering local efforts. 
Chapter 2-funded initiatives now comprise nearly $300,000 of district allocations. 

Despite the positive results which committee members see, summaries of dollars spent and 
students served tell only a portion of the story. As a result, and in response to our 
legislative mandate, the Governor's Advisory Committee embarked upon an Evaluation of 
Program Effectiveness, conducted by the Arizona Department of Education. 

The document for which this letter serves as cover is bounded by research pinpointing 
effectiveness of the largest of the targeted assistance areas: Programs of Training & 
Professional Development. It is hoped that our report will amply reflect the quality of the 
entire Chapter 2 Program and elicit an ongoing dialogue at the state and national levels. 
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Parti 
INTRODUCTION 



A. The Overall Goals of Arizona* s Chapter 2 Program 



With passage in April 1988 of the 
Hawkins-Stafford Elementary and 
Secondary School Improvement 
Amendments, Congress ushered in six areas 
of targeted assistance to education under 
Chapter 2 of the Elementary and Secon- 
dary Education Act (ESEA). 

Entitled Federal, State, and Local 
Partnership for Educational Improvement, 
Chapter 2 solidified a series of robust 
accountability measures. Chief among 



responsibilities under ESEA Section 
1521(aX3): 

• assistance to local educational agencies 
(LEAs) in conducting their Effective 
Schools programs and 

• statewide activities to carry out Effective 
Schools programs. 

Of $783,251 available for state use in SY 91 
(school year 1990-91), the ADE reserved 



Chapter 2 
requires SEAs 
to repori the 
extent of 
services and 
evaluate their 
impact upon 
the state's 
educational 
climate. 



these were the requirements placed upon 56.4 percent of set-aside dollars for the 



state educational agencies 
(SEAs) to report the extent of 
services being delivered and to 
evaluate their impact upon each 
state's educational climate. 

In compliance with ESEA 
mandates, the Arizona Depart- 
ment of Education (ADE) has 
completed its evaluation of 
effectiveness for state and 
locally administered programs 
in the context of Arizona's 
existing Chapter 2 goals. 



The Evaluation of 
Chapter 2 Program 
Effectiveneee follow* 
the.U& Department of 
Education** report 
formerHinp puidanee* 

Am a convenience to 
thoee readers whoee 
intent ie to view more 
quickly the content 
specified in Section 
1522(a)(6)(B) of ESEA 
Chapter 2 9 the 
following parte are 
recommended* 



state's Effective Schools Pro- 
gram. Because of available 
Chapter 2 funding, ADE made 
sizable inroads during the pe- 
riod covered by this evaluation 
in prescribing vigorous pupil 
competency requirements 
through the creation of the 
Arizona Student Assessment 
Program (ASAP). 



This report has been submitted 
for review and comment by the 
Governor's Chapter 2 Advisory 
Committee. Publication of the 
Evaluation of Chapter 2 
Effectiveness is the final event 
in a series of steps taken to make research 
findings and subsequent analyses of 
programmatic impacts available to 
Congress and the general public. 



State Chapter 2 Goals 



In planning annual expenditures tied to 
an Effective Schools effort, the ADE 
exceeds the requisite spending thresh- 
old established by Congress, fulfilling 



Requirement* related 
to publication of thie 
report are addreueed 
within Table 1; Part 
LC; Parte ILC, ILD and 
ILE; an well at 
Appendices 1 and 2* 



This innovative endeavor, 
under its operational umbrella, 
Goals For Educational 
Excellence, represents 
Arizona's primary response to 
the national Effective Schools 
mandate. Thus, ASAP serves 
as the springboard for 
fundamental shifts in student 
assessment and reform of 
instructional delivery methods. Successful 
implementation of ASAP depends upon a 
restructuring of curriculum against a list of 
requisite basic and higher-order skills, the 
mastery of which the State Board of 
Education considers essential to learning 
in grades K-12. 

To ensure successful implementation of 
ASAP and thereby realize a majority of its 
Chapter 2 goals, the ADE launched a 
transitional program of Training & 
Professional Development (T&PD). 



To gain an 
appreciation 
of local 
Chapter 2 
goals in terms 
of dollar 
commitments, 
see Table 1 on 
page 4. 



The ADE provided these Chapter 2- 
supported training services to represen- 
tatives of Arizona public school districts, as 
well as ADE liaisons who worked as 
technical advisers in initiating ASAP. 

Viewed solely in terms of progress made in 
SY 91 , this state-sponsored initiative came 
to be seen as a dominant vehicle for driving 
certain local educational innovations and 
systemic reforms. In early planning of this 
evaluation and as evidenced by findings 
presented in Part IIA, researchers found 
many reasons for including ASAP-related 
T&PD activities within the Evaluation of 
Chapter 2 Program Effectiveness. 



Local Chapter 2 Goals 
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Each year the Arizona Department of 
Education receives federal funding under 
Chapter 2 that is delivered in the iorm oi 
financial assistance to the state's local 
educational agencies (LE As). By restricting 
requirements imposed upon public school 
districts to those found solely within 
regulations promulgated under ESEA, the 
ADE assists in promoting its overall goals 
of 

• increased flexibility in a district's choice 
of allowable activities, 

• significant reductions in local 
administrative burdens, and 

• full access by private school students to 
services. 

To gain a better understanding of specific 
goals inherent in Arizona's local Chapter 2 
programs, the reader is invited to examine 
Table 1, Three-Year Summary of Targeted 
Assistance Areas, on page 4. This 
information further clarifies Chapter 2 
spending trends. 

ADE's process for Special Projects 
applications, the mechanism used to allocate 
local Chapter 2 funds, characterizes the 
steps that must be followed to ensure 



systematic consultation between parents, 
educators and the community at large. 
Illustrative of increased planning and 
consensus-building during each of three 
school years, SY90, 91 and 92, are the 
dramatic, ten-fold increases in dollar 
commitments to locally administered 
Effective Schools programs. 

State goals encourage innovation and 
require the tracking of improvements in 
student achievement whenever LEAs plan 
Chapter 2-supported acquisitions. For 
example, purchases of library books and 
reference materials under Targeted 
Assistance Area 2a must support academic 
pursuits in literacy, math, science and other 
realms. Refocused and far from a concept of 
buying things, 2a allocations have 
diminished 21.8% over three years. By 
contrast, procurement of classroom 
computers accelerated by 47.2%, suggesting 
the shifting emphasis and true impact 
districts ascribe to instructional technology. 

The most compelling example of 
programmatic thrust among LEAs is their 
11 percent increase, over three years, in 
funding for local programs of Training & 
Professional Development. 

Throughout the twelve months covered by 
Arizona's evaluation, 66 districts spent more 
than $2.1 million on Chapter 2-supported 
T&PD activities. Seen purely in terms of 
goal-setting, this sizable amount represents 
38.9 percent of SY 91 dollars distributed to 
Arizona's 197 participating districts. 

Comparable to assertions made at the state 
level, local programs of Training & 
Professional Development are recognized 
as a key element of systemic change. 
Findings presented in Part II.B reinforce 
the notion that T&PD activities 
encompassed a majority of the Chapter 2 
goals set at both local and state levels while 
providing the critical ingredient for 
educational restructuring and offering 
assurance that all children have access to 
the finest educational practices. 
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B. Allotment and Procedures for Allocation of Chapter 2 
Funds 
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Arizona's allotment ixnder Chapter 2 is 
.based upon the state's population of 
pupils, age five through seventeen. The 
ADE received $6,232,511 in SY91, the 
period during which this evaluation was 
conducted. Approximately 13 percent of 
that amount was reserved for state- 
initiated projects and leadership activities, 
including the allowable portion covering 
program administration. 

As previously mentioned, more than 56 
percent of the $783,251 reserved for state 
use was applied to the ADE's Effective 
Schools effort, including those Training & 
Professional Development activities that 
assisted local educational agencies in 
implementing the Arizona Student 
Assessment Program. 

The ADE distributeda little over 87 percent 
of the state's allotment to LEAs in SY 91. 
$5,449,260 was allocated to 220 school 
districts and five accommodation districts 
to address the supplementary educational 
needs of 588,648 public school students 
and 20,632 private school students in 
grades K-12. 

One hundred ninety-seven public school 
districts, among the 225 potential applicants 
for funding, submitted their SY 91 Chapter 
2 applications. Districts participating in 
the program were awarded 99.3% of all 
available dollars. LEAs which chose to 
forego Chapter 2 funding despite an 
apparent need did so because of extremely 
small allocation amounts. 

Allocation Formula. The allocation 
formula established by the Governor's 
Chapter 2 Advisory Committee and 
approved by the Secretary of Education is 
governed by the following criteria. 



Enrollments - LEAs are credited with 
student counts for the school year (July 1 
through June 30) which precedes the t wel ve- 
month period for which the allocations are 
being made. Verified 40th-day student 
counts are used for both public and private 
schools participating in programs assisted 
under Chapter 2. 

T fi ? h Cost Populations - (1) LEAs are 
awarded an additional .05 weight for 
"children living in areas with high 
concentrations oflow-income families"based 
upon at least 49 percent of the student 
population qualifying for the federal 
program of free/reduced lunches; (2) LEAs 
with "children from tow-income families" 
receive an additional .05 weight for each 
Chapter 1-eligible child; and (3) LEAswith 
enrollments of less than 500 students are 
awarded a weight of .1 per student to offset 
problems encountered by "children living 
in sparsely populated areas." A unified 
district in Arizona whose K-8 or 9-12 
enrollment is below 500 students also 
qualifies for this additional funding 
increment. 

During SY 91, approximately $5.5 million 
was allocated at the local level for all six of 
the targeted assistance areas specified 
under Chapter 2, Section 1531. Table 1 
identifies amounts budgeted by districts 
within these program areas and gives the 
number of districts operating such 
programs. Chart 1 displays a bar-graph 
representation of impacts of local decision- 
making during SY 90, 91 and 92. Clearly, 
there are many Arizona districts which 
foster innovative classroom practices 
through programs of Training & 
Professional Development and acquisitions 
that are triggered by the infusion of 
updated curricular content. 



Ofall 
Chapter 2 
dollars 
available for 
allocation at 
the local level, 
99.3% 
reached 197 
participating 
districts in 
SY 91. 



Table 1 

Three-Year Summary of Targeted Assistance Areas 



Chapiar 2 Targeted 
Assistance Area* 


Amount 
Budgeted 


SY 90 

Percent 
ofToUl 


No, of 
Districts 


Amount 
Budgeted 


SY91 

Perorut 
ol Total 


No. of 
Districts 


Amount 
Budgeted 


SY92 

Percent 
of Total 


No, of 
District* 


1. JrrogTfiuTiB Bcrvijig Btuuents at 




















risk and thoee whose educa- 




















tion entails higher costs 


$ 707,590 


13.3% 


34 


$ 636,409 


11.8% 


51 


$ 548,032 


9.8% 


44 


2. Programs to acquire and use: 




















a. Library Materials 


807,783 


15.2 


119 


794,613 


14.7 


118 


632,022 


11.3 


106 


d. L/ompuiers/ooivware 


Jul ,U**0 


6.1 


58 


344,479 


6.4 


68 


473,476 


8.5 


62 


c. Other Curncular Matcnals 


300,657 


5.7 


no 
OO 


07O 1 DO 






001 107 


0.0 


59 


3. Innovative programs: 




















a. Schoolvndc Improvements 








_ 




_ 


9,097 


.2 


2 


b. Effective Schools Program 




_ 


mm 


25,759 


.5 


1 


273,845 


4.9 


4 


4. Programs of Training & 




















Professional Development 




















(T&PD) 


1,915,976 


36.1 


63 


2,105,389 


38.9 


66 


2,127,466 


38.2 


79 


5. Programs enhancing personal 




















excellence and student 




















achievement, including: 




















a* Ethics 














1,000 


.1 


i 


b. i orforming & Creative Arts 




9 7 


19 


126 185 


2.3 


19 


126,867 


2.3 


21 


c. Humanities 








ft 4.00 


H 


4 


5,700 


.1 


4 


cl i^nysicai r luiess 






2 


4,343 


.1 


4 


2.810 


.1 


4 


c. Comprehensive Health 




Q 
.O 


i 
i 


45,812 


.8 


2 


39,788 


.7 


2 


f, Community Service 








6,210 


.1 


2 


7,337 


.1 


4 


g. Other 


286,033 


5.4 


12 


234,564 


4.3 


16 


199,610 


3.6 


32 


6. Programs enhancing school 




















climate and educational 




















programs, including: 




















a. Gifted & Talented 


154,555 


2.9 


23 


141,301 


2.6 


25 


145,226 


2.6 


23 


b. Technology Education 








2,500 


.1 


1 








c. Early Childhood Education 


83,750 


1.6 


2 


94,205 


1.7 


6 


83,608 


1.5 


8 


d. Community Education 


54,378 


1.0 


5 


31,360 


.6 


8 


36,006 


.6 


6 


e. Youth Suicide Prevention 




















f. Other 


295,247 


5.6 


31 


181,780 


3.4 


25 


243,553 


4.4 


29 


7. Program Administration 


192,032 


3.5 


32 


256,159 


4.6 


34 


297,573 


5.2 


42 


TOTAL 


5,304,530 






5,411,650 






5,574,153 







Source: Arizona Department of Education Chapter 2 Office. Anminl Special Projects Applications: School Years 89/90, '91 and '92. 
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C. Plan For Selection and Evaluation of Programs of 
Training & Professional Development 



The national Chapter 2 Steering 
Committee heard presentations at 
conferences over the past three years which 
strongly suggested that very few state 
educational agencies possessed the 
resources needed to conduct in-depth, 
formal evaluations of all six targeted 
assistance areas. 

In their attempts to explain the critical 
aspects of programmatic impact and 
effectiveness, SEAs were cautioned not to 
extend their arena of study beyond that 
which might render them incapable of 
offering any but the most topical of 
assertions. 



Arizona's intent was to standardize the 
survey protocols at each of the two levels 
comprising its evaluation. This choice 
helpedunify focus, sampling strategies and 
data collection methodologies while 
improving each survey's relevance to the 
overall investigation scheme. 

Mindful of time and resource constraints, 
the Arizona Department of Education 
chose to limit its evaluation of targeted 
assistance areas to one or two major 
programmatic initiatives being operated 
inSY91. 



Targeted 
Assistance 
Area 4 
accounted for 
the largest 
combined 
allocations 
daring the 
first three 
years of 
ESEA 
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Chart 1 

Three-Year Comparison of Targeted Assistance Areas 
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f Education Chapter 2 Office. Annual Special Project. Application.: School Yean, 89/90. 90/91. 91/92. 
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Qualitative 

summaries of 

various 

targeted 

assistance 

areas 

presented on 
this page help 
to acquaint 
the reader 
with the 
rationale 
behind the 
ADE's 
selection of 
Chapter 2- 
supported 
T&PD 
activities for 
evaluation. 
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Targeted Assistance Area 4, comprising 
programs of Training & Professional 
Development (T&PD), emerged as the only 
area meeting criteria established at the 
onset of planning. This one program area: 

• accounted for the largest combined state 
and local allocations, 

• involved a substantial degree of 
participation by Arizona districts, 

• produced the greatest impact upon 
educators and students, and 

• embodied shared goals and focus at both 
state and local levels. 

In reflecting upon ADE's decision to restrict 
the scope of the evaluation by narrowing its 
selection of targeted assistance areas to 
Area 4, it is helpful to examine the inherent 
difficulties linked to possible evaluation of 
the otner live areas. 

Targeted Assistance Areas 1, 5 and 6. 

Three of the six targeted assistance areas 
consist of instructional activities that offer 
services directly to students. Although 
significant at the local level in terms of 
Chapter 2 dollars allocated (see Table 1), 
state-level expenditures in Targeted 
Assistance Areas 1, 5 and 6 are relatively 
small. 

Attempts at the local level to assist ADE in 
evaluating these areas were met by 
predictable obstacles. Notable was the 
expressed inability of districts to design 
and implement pre- and post-assessments 
of student gains attributable to the 163 
different programs and to complete this 
work within the allotted time. 

Researchers felt that the flexibility 
employed in designing curriculum and 
programs in participating districts 
precluded development of uniform 
measurement pn>cedures. In addition, the 
potential for gaps in individual student 
achievement indicators over a multi-year 
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period pointed to anticipated shortfalls in 
data collection which would later skew 
results and influence statistical reliability. 

Targeted Assistance Area 3. Chapter 
2-controlled expenditures under Targeted 
Assistance Area 3, while growing 
dramatically, had not exceeded five percent 
of all local allocations during the first three 
years ofESEA. 

Most local Effective Schools programs and 
schoolwide improvement activities, 
especially in urban districts with 
enrollments of more than 5,000 students, 
had been established using nonfederal 
funds. As a result, LEAs which operated 
their locally supported adaptations of the 
national Effective Schools model in SY 91 
were not eligible for consideration in this 
evaluation. 

Targeted Assistance Area 2. 3EA 
expenditures for library books, reference 
materials, classroom computers, 
instructional software and other auricular 
materials were virtually nonexistent in 
SY 91. At the local level, however, these 
important purchases made up the second 
largest co mmi tment of Chapter 2 funds. 

Targeted Assistance Area 2 remained for 
some time a viable candidate for evaluation 
because LEAs consistently document the 
extraordinary need for such expenditures. 
Districts continue to view this program in 
terms of its flexibility and responsiveness 
to the decision-making prerogatives 
entrusted to classroom teachers. 

However, Area 2 was not selected due 
primarily to a lack of comparable 
expenditures at the state level. Additionally, 
the absence of uniform recordkeeping by 
LEAs would have made collection of 
verifiable teacher and student feedback 
nearly impossible. 

Researchers shared the obstacles 
attributable to investigations of Area 2 and 



other major areas with members of the 
Governor's Chapter 2 Advisory Committee 
at the onset of the planning process. 
Discussions eventually centered upon the 
selection of Training & Professional 
Development (T&PD) activities for the 
state's bilevel evaluation. 

At the state level, researchers devised two 
surveys involving both departmental 
employees and district personnel 
participating in ADE-administered T&PD 
activities that were tied to Arizona's 
Effective Schools effort. 

At the local, level the evaluation design 
called for two additional surveys - one to 
capture feedback from individuals who 
served as district T&PD coordinators in the 
43-district study group and one to cover a 
sample of the more than 5,800 trainees 
involved in district training programs. 

Rationale at the State Level 



The Arizona Department of Education 
budgeted 25 percent, or approximately 
$194,000, of its state set-aside monies in 
SY 91 for ASAP-related Training & 
Professional Development. Of this 
amount, the ADE allocated $144,000 
within its Effective Schools line item to 
provide Arizona educators with the 
knowledge and skills their districts 
would require to implement the Arizona 
Student Assessment Program. 

A majority of LEA representatives 
attended both a fall 1990 ASAP conference 
and one of the 15 regional workshops that 
were held in the spring of 1991. Research- 
ers developed the Survey of Conference 
Effectiveness (see Appendix 3) to gather the 
impressions of these participants and 
request their assessments of the quality of 
state-sponsored training they received. 

The ADE earmarked an additional $50,000 
of Chapter 2 set-aside funds under Tar- 



geted Assistance Area 4 for the training of 
state employees who functioned as district 
liaisons for the ASAP implementation 
process. Because a small number of ADE 
staff were involved, researchers used an 
interview format (see the ASAP Liaison 
Questionnaire located in Appendix 4) to 
investigate the effectiveness of in-service 
training given the 15 ASAP liaisons. 



Rationale at the Local Level 



Selection of Targeted Assistance Area 4 at 
the state level encouraged researchers to 
review more fully the role which programs 
of Training & Professional Development 
play at the local level. 

During SY 91, 66 districts or roughly one- 
third of all participating districts in Arizona 
allocated Chapter 2 funds for T&PD 
activities. Although a greater number of 
districts chose to fund more sufficiently 
other program areas, Targeted Assistance 
Area 4 remained significant in terms of its 
monetary impact. A comparison of all 
allocations (refer to Table 1) indicated that 
T&PD expenditures accounted for the 
largest dollar totals in each of the three 
years following ESEA authorization. 

Having established the rationale for 
including locally administered T&PD 
activities in the state's evaluation, the next 
phase of the planning process required 
identification of a viable study group at the 
LEA level. Among considerations was the 
extent to which Chapter 2 funds were being 
applied in terms of program design and 
district control. 

Earlier inquiries of district contact persons 
indicated, especially among larger LEAs, 
that local T&PD matching resources more 
than doubled whenever Chapter 2 funding 
rose above an observable threshold of 
$5,600. 



Classroom 
teachers are 
frequently the 
first to 
recognize the 
long-term 
benefits of 
staff develop- 
ment 



Reasonable prospects for limiting the size 
of this emerging study group revolved 
around the fixed $5,600 cutoff. The 
primary intent was to drop from formal 
evaluation any incidence of Chapter 2 
funding linked to one-time in-services or 
training not fully identified with an 
ongoing T&PD program. 




county so as to preserve the statistical 
reliability of collected data. 

In contrast to concerns regarding the 
exclusion of some feedback, note that 
researchers barred from scrutiny merely 
1.9 percent of the amount budgeted during 
SY 91 for Chapter 2-assisted T&PD 
activities. More than 98. 1 percent of selected 
T&PD expenditures and the resulting 
impact of these dollars remained available 
for study, although the final LEA study 
group was confined to just 43 of the 66 
participating districts. 

The following data define the study group 
in terms of its proportional representation 
within the larger membership of the 66 
districts operating Chapter 2-supported 
T&PD programs and in relation to the 220 
districts in Arizona which enroll students. 



The survey design incorporated a realization 
that trainees attending Chapter 2- 
supported T&PD activities would possess 
the critical assessment skills needed to 
measure benefits they had derived from 
training. 

As professional educators, their feedback 
on program impacts would, therefore, help 
ensure accuracy of evaluative data. 

Evaluating programs at the local level 
centered on the statistical adequacy of the 
emerging study group. There was an 
underlying rationale for restricting the 
size of this group. Researchers felt that 
trainees from districts with greater dollar 
commitments to training would find the 
msgority of questions highly relevant and 
easy to answer. Yet, the study group had to 
be large enough to ensure responses in 
sufficient numbers, termed a valid N 



Relative Characteristics of the 
43-District Study Group 


Category 


Study 
Group 
Total 


As a % of 
66 

Districts 


As a % of 
Arizona 
Districts 


Students 


476,645 


94% 


73% 


Schools 


613 


90% 


58% 


Total 

Certified 

Staff 


28,199 


93% 


70% 


Adminis- 
trators 


1,387 


91% 


65% 


Teachers 


24,575 


93% 


71% 


Support 
Services 
Staff 


2,237 


91% 


65% 



As shown, the study group comprised 43 
districts which included 73 percent of 
Arizona's overall student enrollment and 
70 percent of all certified personnel 
employed in public schools. This group also 
encompassed 58 percent of the state's public 
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schools and, most significantly, 71 percent 
of our K-12th grade teachers. 

The Chapter 2 Coordinator Questionnaire 
placed a strong emphasis upon the 
impressions of 43 individuals who served 
as district T&PD coordinators. Researchers 
developed this instrument to investigate 
the structure and administration of T&PD 
activities within the study group. 
(Appendices 5 and 6 contain the two survey 
instruments used to evaluate locally 
administered T&PD programs.) 

This discussion of Arizona's Evaluation Plan 
concludes with a notable assertion: selection 
rationale at both state and local levels rested 
heavily upon the extent to which LEAs 
embraced T&PD activities as their primary 
means of influencing structural change and 
educational reform. 

The magnitude of potential impact upon 
the state's educational community, in terms 
of the number of students and public schools 
within the study group, is suggested by 
Table 2, Characteristics of Districts with 
Chapter 2-Funded T&PD Programs. 

To contrast the 43 districts in the study 
group with theremaining 23 districts which 
also participated during SY 91 in Chapter 
2-funded staff development, refer to 
Figure 1, Dispersion of Locally 
Administered Programs of Training & 
ProfessionalDevelopment.Figure 1 displays 
the location of each of the 66 public school 
districts and illustrates the acceptance that 
T&PD programs have gained throughout 
Arizona. 4 
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Table 2 

Characteristics of Districts with Chapter 2-Funded T&PD Programs 



District* Within Study Group (43) 


Administrator* 


Teachers 


Support Svcs. 


All Certified 


¥<S"i tv^n m Art fcn 


IMO. OI 

Schools 


Alhambra Elementary 


22 


420 


22 


464 


• 8,720 


10 


Amphitheater Unified 


96 


726 


102 


924 


15,080 


16 


Apache Junction Unified 


18 


197 


58 


273 


4,386 


5 


Catalina Foothills Unified 


8 


182 


15 


205 


2,579 


4 


Chandler Unified 


42 


598 


77 


717 


12,010 


14 


Chino Valley Unified 


8 


88 


7 


103 


1,521 


3 


Crane Elementary 


13 


244 


6 


263 


5,309 


5 


Creighton Elementary 


15 


300 


28 


343 


5,381 


7 


Deer Valley Unified 


41 


845 


50 


936 


16,833 


18 


Flagstaff Unified 


43 


633 


56 


732 


12526 


18 


Flowing Wells Unified 


20 


254 


20 


204 


5,720 


8 


Ganado Unified 


11 


116 


15 


142 


2,020 


4 


Gilbert Unified 


24 


566 


80 


670 


11,497 


12 


Glendale Elementary 


28 


485 


22 


535 


9,643 


12 


Glendale Union High 


47 


795 


55 


897 


13,162 


9 


Indian Oasis-Baboquivari Unified 


11 


86 


6 


103 


1,007 


4 


Isaac Elementary 


18 


269 


11 


298 


5,568 


6 


Kyrene Elementary 


26 


550 


39 


615 


11,219 


13 


Lake Havasu Unified 


14 


213 


10 


237 


4,494 


6 


Laveen Elementary 


7 


108 


6 


121 


1,753 


2 


Litchfield Elementary 


5 


73 


3 


81 


1,561 


3 


Madison Elementary 


16 


215 


9 


240 


4,078 


6 


Marana Unified 


26 


374 


26 


s2C 


8,020 


11 


Maricopa Unified 


5 


52 


13 


70 


936 


3 


Mesa Unified 


114 


3,292 


212 


3,618 


67,695 


62 


Page Unified 


9 


177 


32 


218 


2,960 


4 


Paradise Valley Unified 


56 


1,489 


82 


1,627 


28,657 


30 


Pendergast Elementary 


15 


235 


11 


261 


4,364 


5 


Peoria Unified 


60 


1,153 


131 


1,344 


21,900 


23 


Phoenix Elementary 


29 


442 


14 


485 


8,509 


16 


Phoenix Union High 


59 


1,194 


111 


1,364 


21,495 


16 


Roosevelt Elementary 


48 


584 


25 


657 


11,228 


18 


Scottsdale Unified 


53 


1,161 


70 


1,284 


21,076 


25 


Sierra Vista Unified 


no 

2o 


344 


ZD 


OfiO 


t ,Uoo 


Q 
O 


Sunnvside Unified 


35 


697 


58 


790 


14,210 


17 


Tempe Elementary 


52 


786 


51 


889 


11,056 


23 


Tcmne Union Hicrh 

J. will W lllvll Ail^ll 


26 


584 


155 


765 


8,916 


4 


Tolleson Union High 


18 


155 


12 


185 


2,603 


2 


Tucon Unified 


178 


2,805 


468 


3,451 


60,556 


108 


Washington Elementary 


58 


1,287 


46 


1,391 


23,8i?4 


32 


Williams Unified 


3 


45 


3 


51 


747 


3 


Yuma Elementary 


31 


429 


15 


475 


9,203 


15 


Yuma. Union TTiph 


17 


293 


26 


336 


6,608 


3 


otuay Uroup lotals ax al 


1 A A Q 

l,44o 


OK KA 1 


Z,Zo4 


no o<7q 




olo 


% of RhiHv & Nnnatnidv frTOimR 


91% 


93% 


91% 


93% 


94% 


90% 


% of all Arizona Districts 


65% 


71% 


65% 


70% 


73% 


58% 


Study Group Average 








681 


11,576 


14 


Nonstudy Group Totals SY 91 


147 


1,938 


231 


2,316 


32,832 


66 


% of Study & Nonstudy Groups 


9% 


7% 


9% 


7% 


6% 


10% 


% of all Arizona Districts 


7% 


5% 


7% 


6% 


5% 


6% 


Nonstudy Group Average 








101 


1,427 


3 


Total of Study & Nonstudy Groups 


1,595 


27,479 


2,515 


31,589 


530,597 


679 


Total of all Arizona Districts 


2,242 


35,929 


3,535 


41,706 


683,876 


1,059 



Source: Research and Development, Arizona Department of Education, School District Employers Report, School Year 1991. 



Figure 1 

Dispersion of Locally Administered Programs of Training & Professional Development 




Sixty-six local educational agencies (LEAs) in 
Eleven Arizona Counties operate Chapter 2-fundcd 
Programs of Training & Professional Development 

Source: Arizen* Department of Education, Chapter 2 Unit, Annual Special Project! Applications, School Year 1991. 
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Part II 

EVALUATION OF 
CHAPTER 2 PROGRAM 
EFFECTIVENESS 



Overview. This portion of the report 
describes and explains the four 
separate, but interrelated, areas of 
investigation making up the Chapter 2 
evaluation. To gather data, the Arizona 
Department of Education (ADE) Research 
and Development Division initiated two 
surveys at the state level and two at the 
local level. The framework for Arizona's 
evaluation effort is formed by these survey 
instruments. 

Sections A and B clarify protocols for 
individual surveys in relation to focus, 
sampling strategy and data collection 
methodology. The specific findings reported 
for each of the four surveys are discussed in 
terms of their significance within the 
overall evaluation scheme. Part II 
concludes with several interrelated 
perspectives, namely sections C, D, and E, 
which highlight notable changes in 
educational services, effects upon Arizona's 
students and teachers, and effectiveness 
indicators pertinent to this evaluation. 

State-administerad Training & Professional 
Development (T&PD) encompassed a mqjor 
statewide conference and regional 
workshops for local educational agency 
(LEA) staff. All these activities were 
conducted during the SY91 evaluation 
period. Also included were orientation 
sessions delivered to ADE employees 
assisting school districts in their pursui t of 
strategies conducive to implementing the 
Arizona Student Assessment Program 
(ASAP). 

At the state level, the Survey of Conference 
Effectiveness uncovered impacts 
experienced by local district representa- 
tives who participated in the fall 1990 ASAP 



conference and follow-up workshops in 
spring 1991. A second survey, the ASAP 
Liaison Questionnaire, gauged the 
effectiveness of training and orientation 
given to ADE staff working as technical 
liaisons to facilitate the state's inauguration 
of ASAP. 

Together, these surveys performed the vital 
role of investigating the capacity of ASAP- 
related Training & Professional Devel- 
opment to foster the establishment of this 
statewide initiative for educational 
innovation and systemic reform. 

In similar fashion, two complementary 
surveys were utilized to appraise locally 
administered T&PD activities in light of 
their potential to further educational 
restructuring and to assure access for 
children to the finest educational practices. 
The Local Chapter 2 Evaluation Survey 
asked Arizona's educators to assess the 
effectiveness of local T&PD programs 
while a second survey, the Chapter 2 
Coordinator Questionnaire, examined 
administrative structure and the 
established systems for providing local staff 
with Chapter 2-«upported ataff development 
opportunities. 

Part ILB explains the portion of the 
Chapter 2 evaluation dedicated to study at 
the local level. Its contents are pivotal to 
gaining an understanding of the signifi- 
cance attached to the entire report* 

Analysis of locally administered T&PD 
programs yielded a wealth of insights into 
what is undoubtedly the dominant vehicle 
for effecting excellence among Arizona's 
teachers, administrators and support 
services personnel. $L 



To evaluate 
the effective- 
ness of 
Arizona's 
Chapter 2 
Program, 
researchers 
developed 
four 

surveys — two 
at the state 

level and two 
at the local 
level. 
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A. State-Administered Programs of Training & Professional 
Development (T&PD) 



1. State Funds Allocated for T&PD Activities under the 
Arizona Student Assessment Program (ASAP) 



1 



Ihe evaluation of state-administered 
programs of Training & Professional 
Development involved two surveys. First, 
the Survey of Conference Effectiveness 
investigated the impact of the training 
experienced by LEA staff engaged in 
learning about the workings of the Arizona 
Student Assessment Program. This survey 
was distributed to local district 
representatives who participated in one of 
the ASAP regional workshops supported 
by Chapter 2 funding which were held in 
SY91. 

A second survey, the ASAP Liaison 
Questionnaire, probed a disparate but 
complementary aspect of training. ADE 
employees serving in the technical 
assistance role of ASAP liaisons assessed 
the efforts put forth by the state educational 
agency (SEA) to prepare them for 
introducing ASAP on the district level. 

Excluding Chapter 2 expenditures for 
ADE oversight and program adminis- 



tration, $587,439 was available in SY91 Evaluation of 
for a variety of targeted assistance areas. 
Thirty- three percent or $194,000 was 
budgeted for the state-administered 
program of Training & Professional 
Development selected for evaluation. 



Of this amount, approximately $144,000 
was dedicated to the training of local dis- 
trict representatives attending one major 
statewide conference and follow-up 
regional workshops held in support of 
the state's Effective Schools effort, of 
which the Arizona Student Assessment 
Program is an integral part. 

In addition to supporting this statewide 
initiative through the fall 1990 conference 
and spring 1991 workshops, another 
$50,000 was earmarked for the training 
needed by state ASAP liaisons to enable 
them to assist districts in ASAP 
implementation. 



state- 
administered 
T&PD 
programs 
involved two 
surveys. 



2. The Number and Types of Participants Involved 
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During SY 91, state administrators 
first introduced the Arizona Student 
Assessment Program and devised a 
coordinated plan for training local district 
representatives about ASAP. Full 
implementation of the instructional 
aspects of ASAP as well as the testing 
portion of the program is scheduled for 
September 1993. 



Sixteen individuals employed by ADE 
served as liaisons to districts throughout 
the state. These liaisons facilitated imple- 
mentation of ASAFs instructional and 
assessment strategies for classroom use 
and emphasized the necessity for adoption 
of the complete set of essential skills, 
including basic and higher-order skills, 
upon which ASAP is based 
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In October 1990, ADE sponsored a single 
statewide ASAP conference which 
introduced the main elements of the new 
program to local administrators, teachers 
and other educators. ASAP liaisons 
conducted fifteen regional workshops 
during the spring of 1991. These training 
sessions focused upon exercises tied to 
district acceptance and understanding of 
the new assessment and instructional 
program. 

ADE surveyed 388 participants as they 
departed the spring 1991 regional 
workshops. A comparable exit survey of 
participants who had attended the October 
1990 statewide conference was not pos- 
sible. However, researchers recognized 
that many of the participants attending 
spring workshops also would have 
participated in the fall conference. 

Therefore, the Survey of Conference 
Effectiveness was designed to solicit 
responses relative to both of these training 



components. This cross-matching of 
individuals attending ASAP training under 
two distinct formats yielded useful 
longitudinal information on the effec- 
tiveness of state-sponsored training and 
technical assistance that would not 
otherwise have been available. 

Prior to the scheduling and conduct of 
regional workshops, ADE staff underwent 
extensive ASAP in-service training. Topics 
for these sessions included curriculum 
development and alignment, instructional 
practices and content delivery strategies 
as well as briefings on the emerging 
aspects and unresolved problems asso- 
ciated with ASAP. 

For the Chapter 2 evaluation, 16 ASAP 
liaisons were questioned regarding the 
impact training had upon them. The ASAP 
Liaison Questionnaire was the primary 
survey instrument use^ by r^^eajThe^ +o 
conduct interviews and collect information 
from each of these ADE employees. 



3- Survey of Participants Attending Statewide ASAP 
Training 



♦ Focus 

At the state level, the Evaluation of 
.Chapter 2 Program Effectiveness 
focused on the perceived impact of Train- 
ing & Professional Development activities 
supporting ADE's establishment of the 
Arizona Student Assessment Program. 
Researchers investigated all such state- 
sponsored training conducted during SY 91. 
Benefits and effectiveness of training were 
measured in two research components, 
each dependent upon separate, but 
interrelated, survey instruments. 

The Survey of Conference Effectiveness 
examined the attitudes and impressions of 
LEA participants toward an introductory, 



state-sponsored ASAP conference in the 
fall of 1990 and additional regional 
workshops held some six months later. The 
unique nature of the Arizona Student 
Assessment Program strongly influenced 
the development process needed to achieve 
an appropriate LEA training sequence. 
Therefore, a brief explanation of ASAP 
follows. 

ASAP embodies an innovative approach to 
continuous assessment of student 
performance which, in turn, has prompted 
a fundamental shift in the administration 
of achievement testing, a change in 
instructional delivery methods used in the 
classroom and a revision of the curriculum 
being employed to teach Arizona's K-12 
public school students. 

9" 



The testing portion of ASAP represents an 
uncommon departure from traditional, 
nationally normed tests used in Arizona 
since the early 1980s. The performance- 
based assessments, constructed from a list 
of state-approved essential skills, are 
intended to supplement the more general 
multiple-choice tests given annually to all 
students in all grade levels. 

Bringing ASAP on line necessitated the 
creation of a dual testing program wherein 
the Iowa Tests of Basic Skills (ITBS) and 
the Tests of Achievement and Proficiency 
(TAP) would be administered by districts to 
all students in the first semester followed 
by limited ASAP pilot testing in the second 
semester for students in grades 3, 8 and 12. 
This represented a modification in district 
scheduling of student testing which had 
previously been conducted in April. 
Instructional aids for teachers in grades 
1-12 also had to be integrated into the 
curriculum to prepare students aca- 
demically for this new style of assessing 
their achievement. 

Because ASAP embodied such innovation 
and reform, it was crucial that the Survey 
of Conference Effectiveness be designed to 
focus directly on the following elements: 

• the technical knowledge required of 
facilitators to conduct T&PD sessions, 

• the utility of handouts and other 
materials, 

• the effectiveness of formal presentations 
and small-group exercises, 

• the level of understanding acquired by 
local district personnel regarding ASAP 
and its departure from traditional 
student assessment methods. 

The survey asked participating K-12 
educators for evaluations of both the fall 
1990 and spring 1991 ASAP familiariza- 
tion sessions. 

O 
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♦ Methodology 



The ADE Research and Development 
Division (R&D) designed a survey 
instrument to measure the LEA per- 
spective regarding the impact of ASAP- 
related Training & Professional Develop- 
ment provided by the state. As an aside, 
ADE staff were interested in using the 
evaluation results to assist in determining 
the type and amount of technical assis- 
tance that would be required by LEAs to 
complete ASAP implementation following 
the SY 92 pilot testing. 

The Survey of Conference Effectiveness 
was designed to collect data in three parts. 
First, researchers requested respondent 
demographics. In Part 2, questions were 
asked regarding the effectiveness of the 
regional workshop attended by survey 
respondents. Because the Chapter 2 
evaluation was begun after the October 
1990 conference had been held to introduce 
ASAP, Part 3 of the questionnaire asked 
respondents whether they had attended 
that fall conference. If so, they were asked 
to answer questions on its effectiveness 
and outcomes. 

The survey instrument specifically 
examined the effectiveness of the ASAP 
staff facilitators, the quality of ADE- 
developed handout materials, the 
usefulness of small group sessions and 
whether participation by local district 
personnel led to increased understanding 
of the ASAP goals, objectives and the 
required implementation tasks. 

This survey was distributed at the con- 
clusion of each of 15 regional workshops 
held in March and April 1991. A total of 388 
participants completed questionnaires. 
Their responses provided researchers with 
msgor content groupings of collected data, 
representative of the survey's three parts. 



Districts used 
the 

assessment 
materials 
developed for 
tlieASAP 
regional 
workshops to 
convene their 
own follow-up 
sessions 
locally. 




Appendix 3 contains the survey 
instrument, which is outlined below, and 
survey results in their entirety. 



Survey of Conference 
Effectiveness Outline 

Part 1. Respondent Demographics 

a. Participant Characteristics 

b. Educational Attainment 

c. Job Descriptors 

Part 2. Delivery of Today's ASAP Training 

a. ADE StafFServingas Facilitators 

b. Handouts and Presentation 
Materials 

c. Small Group Session 
Effectiveness 

d. An Overall Perspective of the 
Workshop 

Part 3. October 1990 ASAP Conference 

a. Attendance 

b. ASAP Conference Effectiveness 



♦ Findings 



Three hundred and eighty-eight LEA 
educators attending the spring 1991 
regional workshops completed and 
returned survey instruments to ADE. 



A total count of participants for all 15 
workshops is not available to compute an 
accurate response rate. However, ADE 
workshop facilitators estimated that the 
388 respondents represented appioxi- 
mately three-quarters of total workshop 
participants. 

Respondent Demographics. Complete 
descriptive statistics characterizing 
respondent demographics reported for 
Part 1 of the survey are contained in 
Appendix 3 and also are highlighted here. 



Spring 1991 Regional Workshops 
Respondent Demographics 

PARTICIPANT CHARACTERISTICS 
60 percent female 
90 percent white 

74 percent with a bachelor's or roaster's 

degree 
10 percent with Ph D. 
88 percent held degree in education-related 

field 

PRIMARY JOB CLASSIFICATION 
44 percent classroom teachers 
27 percent administrators 
14 percent school principals 

YEARS IN CURRENT POSITION 
13 percent less than 1 year 
36 percent 1 to 5 years 
22 percent 5 to 10 years 
29 percent more than 10 years 

INVOLVEMENT IN SCHOOLWIDE 
IMPROVEMENT 

57 percent not m a site-based school 

improvement team 
89 percent participate in schoolwide 

improvement activities 

OCTOBER 1990 ASAP CONFERENCE 
ATTENDANCE 

63 percent of regional workshop participants 
also attended the fall conference 



As reported, 44 percent of regional work- 
shop attendees were classroom teachers, 
followed by district administrators (27%) 
and school principals ( 14%). The workshops 
were intended to provide training in the 



O 18 

EMC 



9 U 



administration and implementation of 
ASAP tasks and objectives. 

As a result, these job classification statis- 
tics suggests thftt the workshop infor- 
mation also wou d be communicated 
indirectly to the larger cross-section of 
district and school staff, including class- 
room teachers and policymakers in school/ 
district administration. 

Similar to the statistics reported for 
individuals participating in district T&PD 
activities, statistics generated by 
attendees at the ASAP spring workshops 
indicated that more than half (57%) 
were not part of site-based school 
improvement teams. 

However, nearly 90 percent did participate 
in school wide improvement activities. This 
is a favorable statistic in terms of ASAP 
implementation because these individu - 
als are most likely to share the content of 
their training with a variety of additional 
school or district staff members. 

Finally, 63 percent indicated that they also 
had attended the October 1990 ASAP 
conference. These individuals provided the 
information on conference effectiveness 
requested in Part 3 of the survey. 

Delivery of ASAP Workshop Training. 
In Part 2 of the survey, regional workshop 
participants were asked a series of 
questions regarding the delivery of their 
training. Many of the questions requested 
the professional opinion of trainees. 

Responses were expressed in relative 
levels of agreement or disagreement with 
statements written in the affirmative 
which inquired about some aspect of 
individual workshop experiences. 

Four of the domains queried are presented, 
starting in the adjoining column, and may 
be further clarified as the reader reviews 
Appendix 3. 
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Area: 



Workshop Facilitators (ADE 
Staff) 



Domain: Preparation and Facilitation 
(Questions: Part 2.A.1 - 2.A.5) 

Inquiry: Questions focused on the 
facilitator's preparedness to 
conduct the workshop; their 
ability to communicate goals, 
facilitate discussions, answer 
participants' questions and help 
increase understanding of 
ASAP. 

Revults: Overall, 89 percent of the re- 
spondents felt that the facil- 
itators did a good job of 
answering questions and leading 
discussions which helped them 
in their understanding of ASAP. 
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POSITIVE RANGE 


Agree 


Strongly Agree Combined 


54.4% 


34.4% 88.8% 




NEGATIVE RANGE 


Diiagree 


Strongly Disagree Combined 


9.4% 


1.8% 11.2% 


9 
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Facilitated 
discussions 
among a 
small group 

of 

participants 
often helps 
bring 
important 
issues to 
everyone's 
attention. 
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Area: 



Domain: 



Inquiry: 



Revults: 



Workshop Handouts and 
Presentation Materials 



Utility of Materials 
(Questions: Part 2.B.6 



2.B.8) 



Questions focused on the 
usefulness of materials as tools 
which assist in understanding 
required tasks, support ASAP 
activities and help in informing 
other staff members. 

As seen above, 89 percent of the 
respondents gave facilitators a 
very positive rating for per- 
forming their functions well. 
However, over 93 percent 
responded positively to questions 
on the utility of the handout 
materials, as shown in the data 
response figures below. 





POSITIVE RANGE 


Agree 


Strongly Agree Combined 


50.8% 


42.5% 93.3% 




NEGATIVE RANGE 


Disagree 


Strongly Disagree Combined 


6.1% 


0.6% 6.7% 



issues and engage in dialogue 
about ASAP. 

Result*: Overall , more than 86 percent of 
respondents reacted favorably 
to the hands-on tasks and open 
discussion format. However, this 
domain of questions about the 
workshops received the lowest 
Strongly Agree (30.5%) and the 
highest combined negative 
(13.8%) ratings. 

The following domain on the 
general impressions of the 
workshops received the second 
lowest Strongly Agree (33.6%) 
and the second highest combined 
negative (10.9%) ratings in the 
survey. 



1 


POSITIVE RANGE 


Agree 


Strongly Agree Combined 


55.7% 


30.5% 86.2% 




NEGATIVE RANGE 


Disagree 


Strongly Disagree Combined 


12.7% 


1.1% 13.8% 



Area: 



Domain: 



Area: Workshop Small Group 
Discussions 

Domain: Hands-on Tasks and Open Inquiry: 
Discussions 

(Questions: Part 2.C.9 -2.C.12) 

Inquiry: Questions focused on activities 
and format of the small group 
sessions including the 
usefulness of hands-on tasks 
and the opportunity to clarify 



Workshop Overall Impressions 

Workshop Organization and 
Utility 

(Questions: Part 2.D.13-2.D.15) 

Questions focused on how well 
the workshop was organized, 
the suitability of the format 
used to present and discuss 
ASAP issues and the work- 
shop's usefulness in increasing 
understanding of AS/ J* goals, 
objectives and future district 
activities. 
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Result* Overall, 89 percent of respon- 
dents reacted favorably to 
organisation and format suit- 
ability concerns, while nearly 11 
percent expressed negative 
general impressions* 





POSITIVE RANGE 


Agree 


Strongly Agree Combined 


55.4% 


33.6% 8^-0% 




NEGATIVE RANGE 


Disagree 


Strongly Disagree Combined 


1 9.7% 


1.2% 10.9% 



Workshop Results* In evaluating the 
small group sessions conducted during the 
spring 1991 regional workshops, respon- 
dents indicated that they viewed the hands- 
on tasks they were asked to perform and 
the degree to which they were able to 
clarify issues or engage in useful dialogue 
about ASAP less favorably , generating a 
combined negative response of about 14 
percent, then they viewed the effective- 
ness of facilitators and workshop materi- 
als, which received negative ratings of 
11*2 percent and 6.7 percent, respectively. 

However, overall workshop effectiveness 
received an overhwelmingly positive 
response from nearly 90 percent of the 
respondents, indicating that the work- 
shops were well-organized and provided a 
useful forum for increasing under- 
standing of the ASAP goals, objectives 
and implementation requirements. 

Delivery of ASAP Conference 
Training. Participants in the spring 
1991 regional workshops who also had 
attended the October 1990 ASAP confer- 
ence were asked to complete Part 3 of the 
Survey of Conference Effectiveness. 



Questions in three informational domains 
were included. The first domain consisted 
of statements on conference effectiveness 
in terms of providing a clear understanding 
of ASAP responsibilities, including goals 
and objectives of the new program. 

The second domain dealt with the ability 
of facilitators to answer participants' 
questions concerning ASAP. Finally, the 
third domain focused on the usefulness of 
the conference in terms of preparing 
districts for the spring workshops and for 
initiating the implementation of ASAP. 
Descriptive statistics for Part 3 of the sur- 
vey are contained in Appendix 3. Summa- 
ries of statistics by domain are presented 
below. 



Area: Conference Introduction to the 
ASAP 

Domain: Effectiveness of Goals and Tasks 
Information 

(Questions: Part 3.1,3.2 and3,4) 

Inquiry: Questions focused on how 
effectively the fall 1990 ASAP 
conference provided informa- 
tion on ASAP goals and objec- 
tives, the tasks required of 
districts to implement the 
program and the contrast 
between ASAP and traditional 
student assessment systems. 





POSITIVE RANGE 


Agree 


Strongly Agree Combined 


56.1% 


26.4% &0.5% 




NEGATIVE RANGE 


Disagree 


Strongly Disagree Combined 


17.0% 


2.4% 19.4% 



♦ Comments 



Area: Conference Facilitators 

Domain: Preparation and Facilitation 
(Question: Part 3.3) 

Inquiry: Question focused on ability of 
facilitators to answer questions 
on ASAP. 





POSITIVE RANGE 


Agree 


Strongly Agree Combined 


53.2% 


17.4% 70.6% 




NEGATIVE RANGE 


Disagree 


Strongly Disagree Combined 


23.8% 


5.5% 29.3% 



Area: Preparation for Spring 
Workshops 

Domain: Utility and Applicability 
(Question: Part 3.5) 

Inquiry: Question focused on 
effectiveness of the fall 
conference to prepare district 
personnel for the spring 1991 
ASAP regional workshops. 





POSITIVE RANGE 


Agree 


Strongly Agree Combined 


52.C% 


30.0% 82.3% 




NEGATIVE RANGE 


Disagree 


Strongly Disagree Combined 


15.2% 


2.5% 17.7% 



The statistics yielded by the Survey of 
Conference Effectiveness indicated a posi- 
tive trend in benefits derived from T&PD 
activities associated with the Arizona 
Student Assessment Program. Part 3 of 
this survey revealed that the combined 
negative responses generated by the 
October 1990 ASAP conference were 
higher than those for the spring work- 
shops. However, these results were not 
unexpected in light of the nature of the fall 
conference. 

This statewide conference constituted the 
first formal introduction of ASAP to LEA 
staff. As a radical departure from tradi- 
tional achievement testing, ASAP 
engendered extensive alterations in 
testing procedures which permeated 
throughout the iusu actional infrastruc- 
ture of the existing educational system. 

Establishing ASAP instructional methods 
meant realigning curricula and revamping 
student testing methodologies. Because 
the fall conference heralded these changes, 
overall satisfaction with the event was not 
expected to be high. The combined negative 
score of nearly 30 percent given to the 
ADE facilitators on their ability to address 
questions regarding ASAP reflects the 
frustration felt by the LEA participants. 

In responding to the ASAP Liaison 
Questionnaire, the ADE facilitators 
admitted that they did not even know the 
answers to numerous questions because 
the ASAP implementation process was still 
evolving. This fact also accounts for the 
negative reaction of 20 percent of the 
participants to statements about the over- 
all usefulness of the conference in providing 
clear information on the new program. 

The incidence of negative responses 
declined, however, to a range of 7 to 14 
percent for the questions probing the 



effectiveness of the spring workshops. This 
improvement in satisfaction may be 
attributable to the increased availability of 
ASAP information as development 
proceeded. 

During the intervening four to six months 
between the fall 1990 conference and the 
spring 1991 workshops, district 
administrators and classroom teachers 
received an increased amount of more 
specific ASAP instructional material. 

Additionally, the ADE facilitators pos- 
sessed more training, experience and 
general knowledge about the ASAP by 
the time the workshops were held. 
Facilitators had received extensive 
instruction in the areas of cuiriculum 
development, instructional support and 
assessment design under an ADE staff- 
training component which is described in 
the following section. 

The ASAP Liaison Questionnaire, 
discussed extensively on page 23 and 
throughout the rest of Part II.A.4, 
investigated the Training & Professional 
Development activities developed for the 
ADE staff who served as conference and 
workshop facilitators in their capacity as 
the ASAP liaisons to Arizona LEAs. 




During the 
intervening 
months 
between the 
fall 1990 
conference 
and the 
spring 1991 
workshops, 
district 
personnel met 
to share 
ASAP 
instructional 
materials and 
discuss local 
assessment 
activities. 
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4. Survey of Arizona Department of Education Staff 
Serving as Liaisons to Districts under ASAP 



A3 discussed in the previous section, 
.implementation of the Arizona Stu- 
dent Assessment Program required the 
support of many employees of the state 
educational agency (SEA) as well as LEA 
personnel. ADE staff played a crucial role 
in the conceptual development and even- 
tual implementation of ASAP. 

The operational design was developed 
initially during SY91 by staff of the 
Educational Services Division using 
Chapter 2 funding as seed money. When 
specific goals and objectives evolved from 
exploration of this novel approach, the 
scope and parameters of the program 
were determined and agreed upon by 
State Superintendent C. Diane Bishop, and 
the Aju£ administration. The Arizona 
Legislature concurred in these delib- 
erations. As a result, additional ADE staff 
members were assigned ASAP imple- 
mentation responsibilities between July and 
November 1990. 



♦ Focus 



In SY 91, 16 ADE employees from the 
School Improvement Unit were asked to 
serve primarily as liaisons between the 
ADE and local district personnel. Their 
main tasks included assisting LEAs with 
the coordination and planning of ASAP- 
related tasks, developing appropriate 
instructional and assessment strategies, 
and functioning as conduits through which 
ASAP implementation information would 
flow between the state and the school 
districts. 



Development in-services that these ASAP 
liaisons were given to prepare them for 
assuming their unique positions. 

♦ Methodology 

To gain a better understanding of their 
ASAP-related T&PD activities, preliminary 
interviews were scheduled with seven of 
the sixteen ASAP liaisons. During these 
initial discussions, it became clear that the 
type of staff development and in-service 
training which the ASAP staff experienced 
was significantly different in emphasis, 
structure, content and purpose from 
training routinely offered to local district 
administrators, classroom teachers and 
support services staff. The difference 
resulted directly from the distinctive 
nature of the ASAP itself. 

As opposed to teachers or administrators in 
well-defined jobs with clearly delineated 
performance goals, ADE staff responsible 
for assisting with ASAP found themselves 
in the position of both learning about, and 
creating, their duties and performance 
guidelines. No precedent existed in 
Arizona or any other state which could 
provide either functional guidance or 
training models. 

In addition to hi^nhghting their unusual 
job situation, the ASAP liaisons revealed 
that most of their in-service training was 
conducted by other ADE staff. 

That is, the individuals primarily respon- 
sible for the conceptual development of the 
program had also served as in-service 
providers for new ASAP staff members. 
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The focus of this investigation centered on 
the impact and effectiveness of Chapter 2- 
supported Training & Professional 



At the start, the primary in-service goals 
centered on bringing the new staff up-to- 
date with the developmental status and 
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underlying philosophy of the Arizona 
Student Assessment Program. 

Subsequent in-service sessions focused on 
ASAP problem resolution and imple- 
mentation strategies. These brainstorming 
sessions eventually led to a clearer 
definition of ASAP activities and helped to 
determine the knowledge base required to 
enable ADE staff successfully to serve 
districts and schools in the supportive role 
of ASAP liaisons. 

As a result of these preliminary interviews, 
the methodology used to investigate 
ASAP-related T&PD activities was 
altered significantly. Instead of con- 
structing an objective survey instrument 
similar to those developed for use. at the 
local level, researchers would conduct a 
one-on-one interview with each member of 
the ADE ASAP staff. This decision was 
based on four factors. 

First, the nature of the in-service training 
received by ASAP liaisons differed 
significantly from the LEA model wherein 
T&PD activities offered clearly defined sets 
of skills or knowledge that were to be 
communicated to the participants. While 
the formats employed for local T&PD 
sessions varied among workshops, seminars 
and conferences, the format used for 
training the ASAP liaisons was based upon 
small group discussions. 

Second, the participants in the ASAP 
T&PD activities were, for the most part, 
the same each time. Within LEAs, T&PD 
activities were not confined to a common 
group of individuals with similar job 
descriptions. 

Third, the content of the ASAP in-services 
always concerned the same subject, the 
ASAP initiative. In contrast, T&PD 
activities offered at the district and school 
level spanned a wide variety of content 
areas. F inall y, the number of individuals 
involved in state ASAP staff training was 
very small when compared to the number 



of LEA staff participating in state- 
sponsored T&PD ac ivities. 

A three-part outline of standardized 
discussion questions (see Appendix 4) was 
constructed as the survey instrument from 
which the interviews could be structured. 
In the first part of the formal interview, 
survey questions focused on the learning 
process ADE staff experienced during their 
initial ASAP training as well as on the 
adequacy of the training designed to prepare 
them for their role as ASAP liaisons. 

The second portion of the interview dealt 
with the spring 1991 regional workshops, 
questioning the effectiveness of the ADE 
staff persons conducting the sessions and 
the degree of success achieved in meeting 
the primary goals and objectives of these 
workshops. The fl^nl part of the interview 
centered on the individual's role in the 
October 1990 ASAP conference and 
questioned the success of that conference in 
meeting its established goals and objectives. 



♦ Findings 

ASAP In-service Training. Most of the 
ASAP liaisons who were interviewed 
described their role as one of assistance to 
LEAs in establishing ASAP implementa- 
tion plans. To fill this role, they were 
required to attend ASAP in-service train- 
ing primarily conducted in-house by ADE 
staff. The only facilitator for this training 
was the state ASAP director , who had been 
instrumental in developing the program's 
main goals and objectives. 

Due to time constraints which hurried the 
ASAFs developmental pace, ADE staff 
were required to begin implementation at 
the district level while undergoing train- 
ing to learn how to accomplish this feat. 

ASAP liaison in-services w^re scheduled at 
regular intervals. During these training 
sessions, staff were guided through various 



aspects of the ASAP, such as reviewing 
Arizona's essential skills, curriculum 
alignment strategies, scoring rubrics 
associated with the new student 
assessments and determining scoring 
criteria through the selection of anchor 
papers. 

An important finding demonstrated by 
interview accounts was that the in-service 
training did not, in and of itself, constitute 
the primary source of the ASAP knowl- 
edge and skills required by staff to perform 
their duties. 

Additionally, because refinements to the 
ASAP program were to be made by the 
same individuals who were attending the 
Chapter 2-supported staff training, 
definitive answers and guidance for many 
aspects of the ASAP program were not yet 
available. Consequently, the structure of 
the in-service sessions fostered collective 
discussions from which conserve wa* 
reached upon solutions to problems 
arising from implementation issues which 
had to be officially defined and clarified* 

The result was that all ASAP liaisons 
participated in an evolutionary process 
concerned with delineating, refining and 
solving a myriad of unanticipated prob- 
lems. Selected summaries of interview 
responses to the ASAP Liaison Ques- 
tionnaire appear in the preceding column 
and complete results are included in 
Appendix 4. 

Training Process in General* The first 
interview questions put to ASAP liaisons 
attempted to ascertain their overall 
impressions of in-service training 
developed and provided by ADE. 

Adequacy of Training* The first portion 
of the interview also questioned whether 
the training ADE staff received had 
adequately prepared them for their role as 
ASAP liaisons. Their responses were not 
uniform. Some responses suggested the 
formal in-services only provided the very 
minimum of information and that the real 



learning came from discussions and reading 
outside of the training sessions. 



Comments on the Process for 
Training ASAP Liaisons 

► Most [of the in-services] presented very 
clearly defined goals and were very efficient 
in providingwhat ADE staff needed to know. 

► The in-services involved learning about the 
basic premises of ASAP and then reading 
and discussing amongst [ourselves] the 
many unanswered questions that still 
needed to be worked out. 

► Monthly meetings were held. There were a 
lot of discussions and questions. We 
practiced scoring the assessments and [then] 
discussed differences in the results. The 
entire process was well-organized. 

► The basic organization was very good. There 
were differences of opinion, but the process 
got us through and provided answers to 
most of the questions. 

► Learning ASAP was a process. At the 
beginning, the ASAP director provided the 
in-services and knowledge of the program. 
However, each ADE staff person had to 
spend a lot of time reading and discussing 
with colleagues what ASAP was all about. 



Others suggested that the in-services 
provided the forum for the knowledge thsy 
acquired and in that way successfully 
prepared them for their duties. A selected 
summary of responses to this area of 
inquiry appears on the next page. 

Throughout the interview discussions, 
mention is made of the need for staff to 
xindertal £ a substantial amount of outside 
reading end research. This was necessary 
because loth trainers and trainees were 
learning as they went along. No similar 
assessment program or prototype existed 
in Arizona or any other state which could 
provide a model or blueprint While there 
was a structure to the in-service sessions 
in a global sense, most details of 
implementation had to be developed 
collectively through staff interaction and 
consensus. 



Adequacy of ASAP Liaison 
Preparation 

► [Regarding the in-services] Yes, ASAP staff 
grew along with the evolution of the ... 
program. 

► Yei, very well. I know for some it did not, 
but for me it worked well. The training gave 
me a clear understanding of ASAP. 

>» The sessions, on assessment scoring were 
exceptional. 

► [I] felt incredibly on my own. [I] had to 
constantly go back and talk to the director 
and review research writings to gain new 
knowledge and new perspectives. However, 
this was necessary because of the new 
territory that was being covered. 

► ASAPmovedtoofast.Itwasnotwellthought 
through. [IT frit that all of the information 
on the ASAP program (status and problems) 
was not being passed along to the ADE staff 
and to the districts. 

>» The structured in-service sessions were less 
important than the peer-to-peer interaction. 



Staff Input on ASAP Development. 

When queried about their role in the 
development of the ASAP, nearly all 16 
ADE staff members indicated that they 
had virtually no role in the initial planning 
and conceptualization. 

Many indicated that this caused problems, 
some of which could have been avoided if 
more input from the ADE program area 
specialists had been requested. A selected 
summary of responses is presented in the 
next column. 

Interestingly, many of those interviewed 
stated that while a sense of exclusion from 
the process was present at the beginning of 
their training, the program later evolved to 
encompass a much more cooperative 
developmental process. 



Liaison Input to ASAP Development 

► None whatsoever. None in development of 
the performance-baaed assessment*. None 
in the essential skills or the ASAP 
philosophy. However, some other ADE staff 
did help with the development of the 
assessments because it was their area of 
expertise. 

> Not much. Directions were always set up by 
the ASAP director. But we needed this type 
of leadership due to the complexity of the 
ASAP program. It could not have been done 
by committee. 

>• Ninety percent ADE administrators, 10 
percent ADE staff. This was the balance of 
input to the development of the ASAP 
program. We had to buy into the vision. We 
were told the vision instead of being asked 
about it. 

► Moderate amount , Partici pation in m eetings 
and discussions really had an impact. The 
input from the districts really had an impact 
as well. 

Lack of participation is not necessarily bad 
however, because a committee process would 
have killed the whole development of the 
ASAP. Decisions needed to be made. This 
has not caused a lack of ownership (on my 
part) toward the program. 

> Initially, there was no input from ADE 
staff. However, this has changed 
considerably because ASAP has changed. 
At the beginning, leadership was necessary 
to get things done. Now, ADE staff have 
considerable input into the program and its 
actual implementation. 



Workshop Preparation and Effec- 
tiveness. In the second portion of the 
interviews, questions were asked con- 
cerning the adequacy of staff preparation 
for the spring 1991 regional workshops. 
These discussions focused not only on the 
perceptions of the ASAP staff facilitators 
on whether all of the workshop goals and 
objectives had been met successfully but 
also on whether participants actually 
received useful information and training 
on ASAP as a result of their attendance. 



Statements made during the interviews 
indicated that ADE staff had mixed feel- 
ings about whether their T&PD activities 
had adequately prepared them for the task 
of hosting these workshops. The sum- 
mary of selected statements presented 
below reflects the general sentiment of the 
16 ADE staff members interviewed. 



>* For the moat part, yes. But ...not necessarily 
from the ADE ASAP in-service training 
process* 



The ADE ASAP staff also were asked if 
they had a clear sense of what their 
responsibilities would be concerning 
conducting the workshops. For the most 
part, their responses indicated that their 
T&PD in-services had provided them with 
a clear agenda and goals prior to ■ ,e start 
of the sessions. 

When asked whether or not they were 
successful in communicating the goals and 
objectives of the workshops and whether or 
not they were successful in meeting these 
goals, staff responded positively in all cases* 
This indicates that while some of the ADE 
ASAP staff had reservations about 
whether they were adequately prepared 



to conduct the workshops, in general they 
felt they did a good job of providing the 
necessary information and assistance to 
the LEA participants. Many statements 
indicated that use of effective handout 
materials and a clearly defined agenda 
coupled with small groups and a lot of open 
discussion between the participants greatly 
contributed to the success of the work- 
shops* This cooperative, interactive style of 
training is in contrast to the classical 
teacher-student model of in-service training 
in which information is only handed down 
from the speaker to the listeners. 

As seen previously, responses to the 
interview questions reflected the 
incomplete, open-ended nature of the ASAP 
initiative. That is, a condition of continual 
development existed as opposed to a process 
of implementing an established set of 
activities, skills and required knowledge. 

An apparent outcome of the interview 
discussions concerning the workshops was 
the fact that the LEA participants were 
undergoing the same type of learning 
experience with ASAP as that in which the 
ADE staff were immersed* 

Indeed, flexibility is a hallmark of the 
ASAP program and districts have great 
leeway to develop their own implementation 
plans as long as the intent of the program — 
which is to instruct and assess students on 
the complete battery of required essential 
skills — is met* 

Effectiveness of ASAP Communi- 
cations. A few of the persons interviewed 
did not feel that the workshops were com- 
pletely successful in communicating all of 
the necessary ASAP information to the 
LEA participants. That is, liaisons believed 
some of the participants left the workshops 
with unanswered questions and were still a 
bit unclear as to what was expected of 
them. The selected statements presented 
below reflect the general comments of the 
interview group. 



Preparation of ASAP Liaisons as 
Workshop Facilitators 

>- No. Workload outside of ASAP duties 
prevented some staff from preparing 
adequately for these sessions. However, the 
feeling was that they were generally well- 
prepared for [the workshop] ASAP duties. 

>» Yes, generally well-prepared to conduct the 
workshop. 

► [I] didn't feel well -prepared psychologically, 
but the materials helped a lot to get through 
the sessions. After the meeting was over, I 
felt very good about it. 



Adequacy of ASAP Information 

> Yea, our group asked a lot of questions. 
Some left very confident while some left 
frustrated that other districts were ahead 
of them. 

> Not totally clear, but more so than if the 
workshops had not been held. Some of the 
participants were being exposed to ASAP 
for the first time. 

> Yes, [instruction on the ] District Assessment 
Plans gave the districts the information 
they needed to take back to their local 
organizations. 

> They left with a clearer understanding than 
when they came in, but it wasn't a perfect 
understanding of ASAP. No one had this . 



Overall, the ADE ASAP staff made very 
positive statements about the handout 
materials prepared for the workshops and 
felt that hands-on activities practiced 
during each session were very useful in 
helping district participants to understand 
elements of the ASAP program more 
clearly. 

Finally, only two of the sixteen persons 
interviewed gave the overall effectiveness 
of the workshops an average rating. All of 
the others felt that the sessions were very 
useful and successful \n achieving what 
they were intended to accomplish. 

October 1990 ASAP Conference. In the 
final portion of the interviews, the ASAP 
liaisons were asked to discuss their roles 
and activities at the October 1990 ASAP 
Conference. Discussion focused on their 
preparation for assisting with the activi- 
ties of the conference, including hosting 
small group discussion sessions, making 
presentations on particular aspects of 
ASAP and answering questions on the 
program. 

As reflected in the selected summary of 
statements presented below, comments 
were mixed concerning the conference 



preparedness of staff resulting from their 
earlier T&PD activities. 



Adequacy of Conference 
Preparation 

>- [I] did not feel ao well-prepared for the fall 

1990 conference an [I did for] the spring 

1991 regional workshops. [The] spring 
workshops gave me more time to become 
knowledgeable about ASAP and its many 
components. 

>- [I] felt more prepared for the fall conference 
than for the spring workshops because the 
breakout sessions [that] some staff led 
concernedtheir own particular contentarea 
specialty. 

► [I] did not feel confident going into the fall 
conference because ASAP had too many 
unanswered questions at the time. 

> Yes, [IJ felt confident in the information and 
what needed to be accomplished. 



While some staff responses were negative, 
most felt that the October 1990 ASAP 
Conference was well-organized and that it 
presented the essential features of the 
ASAP. However, a number of the ADE staff 
believed that LEA participants did not 
necessarily leave the conference feeling 
confident about what was expected of them 
or with complete information. 

Due to the brief time lines associated with 
the ASAP developmental process, not all 
details had been addressed in time for the 
October 1990 conference. Participants 
requesting specific information on 
implementation questions, for example, 
could not be helped because the answers 
were not yet known. Definitive guidance 
was lacking for several aspects of ASAP 
since many associated questions had to be 
defined and clarified before answers could 
be formulated. 



♦ Comments 

As reflected in these interviews conducted 
with ADE ASAP staff, the Survey of 
Conference Effectiveness also revealed that 
LEA personnel who participated in both 
the fail conference and the spring workshops 
felt that the October 1990 ASAP Conference 
was less successful at informing districts 
about the activities required under ASAP 
than were the spring 1991 regional 
workshops. As reported in Part ILA.3, the 
LEA participants also gave generally lower 
approval ratings for the effectiveness of 
ADE staff facilitating the fall 1990 session 
than they did for ADE facilitators of the 
spring 1991 workshops. 

Overall, LEAparticipants gave high positive 
ratings for the effectiveness of the state- 
sponsored training they received in the 
spring workshops and felt that the materials 
and information they acquired could be 
shared with additional district staff persons. 

In contrast to the LEA staff experience, 
interviews with ASAP liaisons concerning 
their in-service training revealed mixed, 
but generally positive, reactions to questions 
concerning the effectiveness of these T&PD 
activities. However, it is clear that the 
circumstances under which the ASAP was 
inaugurated precluded the smooth 
integration of knowledge and skills which 
would have enabled ADE staff to serve 
more effectively as liaisons to the LEAs 
during the introductory stage of the 
program. 

Since this evaluation was conducted in 
SY91 while the ASAP was still under 
development, the question of whether or 
not the ASAP staff in-services provided all 
of the information that staff required to 
carry out their duties is irrelevant. The 
more informative question is whether the 
ASAP in-services enabled SEA staff to 
communicate their ASAP knowledge, albeit 
limited knowledge, effectively to district 
staff. 



It is clear from survey responses, especially 
in terms of voluntary self-evaluations 
shared by the majority of ASAP liaisons, 
thatapoeitive progression occurred between 
June 1990 and May 1991 regarding the 
outcomes of the in-service training delivered 
to these individuals. While many felt 
insecure initially about their ability to 
perform ASAP duties, the ongoing in-service 
sessions evidently provided the ASAP 
liaisons with a steady and demonstrable 
development path towards achieving 
effectiveness and success. A 
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B. Locales- Administered Programs of Training & Professional 
Development (T&PD) 



1. Local Funds Allocated for T&PD Activities 



Evaluation of locally administered 
programs of Training & Professional 
Development encompassed two surveys. 
The Local Chapter 2 Evaluation Survey 
was mailed to educators within the 43- 
district study group (refer to Part LC) who 
attended staff development sessions 
during SY91. This survey investigated the 
training experienced by teachers, 
administrators and support services staff 
while identifying the type and effectiveness 
of content delivered as well as relevant 
instructional outcomes. 

The Chapter 2 Coordinator Questionnaire, 
on the other hand, offered a separate but 
complementary perspective. T&PD 
coordinators within the previously iden- 



tified study group (see Table 2, page 10) 
were queried about the organization and 
administrative procedures of their T&PD 
programs that were supported through 
allocations of Chapter 2 monies. 

Including amounts designated for admin- 
istration of local Chapter 2 programs, 
Arizona public school districts were 
allocated $5,411,650 during the evalua- 
tion period Table 1 on page 5 breaks out 
these local SY 91 allocations and reports 
the proportional share ascribed to all 
targeted assistance areas. 

Nearly 40 percent of available Chapter 2 
funds, or $2,105,389, was allocated to 
Targeted Assistance Area 4, Programs of 



Of the more 
than $5 
million in 
Chapter 2 
funding that 
went to 
Arizona 
school dis- 
tricts in 
SY 91, 40 
percent was 
spent on LEA 
programs of 
Training & 
Professional 
Development 
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Source: Arizona Department of Education Chapter 2 Office. Annual Special Projects Applications; School Year 1991. 



The average 
Chapter 2 
T&PD 

allocation for 
study-group 
districts in 
SY91 was 
almost 
$48,000. 



Training & Professional Development. This 
sizable amount represents a 10 percent 
increase over the prior year's allocations 
by LEAs for Area 4 which points to the 
growing significance attached by the 66 
participating districts to the provision of 
various T&PD activities for their 
personnel. 

One distinction between etudy-group and 
nonstudy-group districts is the size of the 



average allocation of Chapter 2 funds for 
local T&PD programs. The 43 districts in 
the study group received an average 
allocation of $47,847, ranging from a low of 
$5,852 to a high of $428,568, In contrast, 
the group of 23 participating districts that 
were not surveyed had an average Chapter 
2 T&PD allocation of just $1,989, ranging 
from a low of $400 to a high of $5,594. 



2. The Number and Type of District Staff Involved 



The public school districts studied in this 
portion of the Chapter 2 evaluation are 
representative of all 66 districts in Arizona 
which operate Chapter 2-funded programs 
of Training & Professional Development. 

In fact, the districts in the study group 
account for 73 percent of our public school 
students and 58 percent of all schools in the 
state. The study group is comprised of 43 
districts with 43 local T&PD coordinators 
and a universe of teachers, administrators 
and support services staff totaling 5,819 
individuals who attended at least one 
Chapter 2-funded T&PD activity during 
the 1990-91 school year. Random sampling 



of this trainee universe assured research- 
ers that survey responses would mirror the 
points of view held by 71 percent of the 
state's K-12 teachers and 70 percent of all 
certified staff in the study group. 

The evidence of positive impacts upon 
students uncovered by the locr! surveys, as 
well as subsequent findings of T&PD 
effectiveness, gains added weight 
considering that teachers made up over 88 
percent of the sampled universe while 
approximately 6 percent were admin- 
istrators and another 6 percent worked as 
support services staff. 



3. Survey of District Administrators, Instructors, and 
Support Services Staff 



♦ Focus 

At the local level as at the state level 
.having been discussed previously in 
section A, the Evaluation of Chapter 2 
Program Effectiveness focused upon 
Training & Professional Development 
activities. Though complemented by a 
subsequent questionnaire given district 
T&PD coordinators, the Local Chapter 2 



Evaluation Survey gathered the bulk of 
the evaluative data on locally administered 
programs of Training & Professional 
Development funded by Chapter 2 monies. 

Administrators, teachers and support 
services staff who participated in local 
T&PD activities during SY 91 were tar- 
geted due to their apparent expertise in 
day-to-day assessments of programs and 
student performance. 



The ADE Research and Development 
Division staff constructed a survey 
instrument to measure a variety of trainee 
characteristics and to collect specific 
information on the multitude of tra inin g 
courses attended by LEA staff in the 43- 
district study group. Specifically, the sur- 
vey contained the four m^jor articulations 
listed below. 

Respondent Demographics 

• Participant Characteristics 

• Educational attainment 

• Job descriptors 

Teacher Characteristics 

• Number of students taught 

• Subject areas taught 

• Grade levels taught 

• Percent of at-risk students taught 

Individual Course Evaluations 

• Content, format and focus 

• Instructor and presentation materials 

• Implementation of acquired skills 

• Impact on job and students 

Overall Trainee Impressions 

• Reasons for participation 

• Effectiveness of training 



♦ Methodology 

The design of Arizona's evaluation of 
Chapter 2-funded T&PD programs was 
based on two fundamental considerations. 
First, the main: purpose of the evaluation 
was to provide state-level information on 
T&PD programs to the U.S. Department of 
Education. 

This information would be constructed out 
of local Chapter 2 evaluation survey data 
gathered from administrators, teachers 
and support services staff participating in 
Chapter 2-supported staff development 
activities in districts around the state. 
Second, it was decided that the Arizona 
evaluation also should provide infor- 
mation to participating districts on their 
respective programs. 



Sampling Strategy. This second 
consideration demanded that an adequate 
number of individuals be sampled from 
each district individually to ensure that 
statistically valid inferences could be made 
concerning each local program. 

To develop a database of T&PD 
participants at the LEAlevel, district T&PD 
coordinators were asked to compile and 
submit to the ADE a list of all staff persons 
in their districts who had participated in . 
Chapter 2-funded T&PD activities during 
the evaluation period The participants were 
classified according to their primary job 
descriptors (administrator, teacher or 
support services staff). 

Responses to this request documented 5 ,819 
individuals in the 43-district study group 
who bad participated in these activities 
during SY- 91. Five thousand one hundred 
Thirty-nine of these individuals were 
classified as teachers, 334 were 
administrators and 346 were classified as 
support services staff. 

Based on this distribution of the three 
descriptive job categories, it was determined 
that the universe of administrators and 
support services staff was insufficient to 
provide reliable results at the district level. 
Because of this, the sampling process v/as 
divided into two separate procedures. 

First, using the list of participatingteachers, 
a random sample of 35 individuals was 
selected from each district. For districts 
reportinglessthan35 teacher participants, 
all of the individuals were included in the 
study. This resulted in an overall selection 
of 1,212 teachers across the 43 districts in 
the study group. 

With an assured nonresponse level of 
approximately 5 individuals per district, it 
was expected that each district would have 
evaluation results based on approximately 
30 responses for this category. 



Statistically 
valid data 
were collected 
for each of the 
study-group 
districts and 
for the state 
as a whole. 
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Because they contained fewer than 35 
participants per district, all 43 LEA lists 
of administrators and support services 
staff were consolidated into a single state- 
level data set. For each category, a random 
sample of 100 individuals was selected. It 
was felt that this number would provide 
sufficient observations from which 
researchers could formulate valid state- 
level information on these categories that 
would constitute adequate representation 
when combined with statistics for the 
teacher category. 

Since the primary emphasis was on devel- 
op; ng an evaluation of district Chapter 2- 
fiinded T&PD activities for the state as a 
whole, the sample taken from each cat- 
egory was compared to the distribution 
initially supplied by the districts. The com- 
parison follows. 



Based on the distributions obtained from 
the sampling process, it was felt that the 
state-wide statistics were reflective of the 
population from which they were drawn. 
In addition, breakouts by job category 
also contained enough observations to allow 



inferences on all participants. However, 
the detailed imports constructed for each of 
the 43 districts are based only on teacher 
responses due to the much lower number of 
observations gathered for administrators 
and support services staff. 

Survey Distribution and Return* 
After development of the LE A-participant 
database and determination of the 
sampling process, ADE researchers 
designed the survey and worked in concert 
with district Chapter 2 T&PD coordinators 
to distribute the survey instruments. 

The sample database that was then 
developed was used by both the ADE staff 
and the T&PD coordinators to track the 
distribution and return of survey 
instruments. At the district level, each 
T&PD coordinator was responsible for the 
distribution and collection of surveys to 
targeted t^i^oA«? in?h ir ^.T>g follow-up of 
surveys not returned by the assigned 
deadline. 



♦ Findings 

The coordinated survey distribution, 
collection andverification process described 
above resulted in a 100 percent response 
rate for the study-group districts. That is, 
all 43 districts submitted at least one 
completed survey to the ADE. 

Tracking within each district, by individ- 
ual, of survey distribution and comple- 
tion, resulted in the return of 975 of 
1,212 survey instruments distributed to 
district administrators, teachers and sup- 
port services staff. 

This cooperative effort generated an 
overall survey response rate of 69 
percent. Only six of the 43 districts had 
response rates less than 50 percent. These 
return rates by job classifications were 57 
percent for teachers, 71 percent for 
administrators and 53 percent for support 



Universe of Eligible Staff 


Category 


Total 


Percent 


Teachers 


5,139 


88.3% 


Administrate r* 


334 


5.7% 


Support 






Services 


346 


5.922 


Total 


5,819 


100.0% 


Sample Selected from 


Study Group Universe 


Category 


Sample 


Percent 


Teachers 


1,!>12 


85.8% 


Administrators 


101 


i.2% 


Support 






Services 


_aa 


7.0% 


Total 


1,412 


100.0% 
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services staff. Twenty-nine surveys were 
returned to the ADE with no job identifier. 

Complete survey information is pre- 
sented in Appendix 5* Major highlights 
are reported on the pages which follow. 

Respondent Demographics. Part 1 of 
the survey covered participant character- 
istics. As shown below, more than three- 
quarters of the respondents were female, 
89 percent white, 93 percent held a 
bachelor's or master's degree and over half 
had been employed for 5 or more years in 
their current positions. 

Participant Characteristics 
975 Total Respondents 
76% Female 
89% White 

93% With Bachelor's or Master's 
Degrees 

55% Employed in Current Position K 
or More Years 

When respondents were asked about 
participation in site-based or schoolwide 
improvement activities, 62 percent 
indicated they were not members of site- 
based improvement teams but 71 percent 
did participate in schoolwide improvement 
initiatives. 

Demographics associated only with 
responses from the teachers follow. 

Teacher Characteristics 

• Average Class Size: 25 

• Grades Taught: 

63% Preschool - giade 6 
37% Grades 7 - 12 

• Teaching Experience: 
58%reportedteachingmorethanl0years 
21% reported teaching less than 5 years 

• Subject Areas taught: 

45% General subjects, Preschool through 
grade 6 

13% Math and science 
14% Language arts 



To understand more about classroom 
characteristics in terms of the students 
being taught, teachers were asked if at 
least 50 percent of their pupils could be 
considered at risk (students having char- 
acteristics recognized as increasing the 
likelihood that they will drop out of the 
educational system), In response, one- 
third (33%) of the teachers indicated this 
was true. (See Appendix 4, Disaggregated 
Data.) A discussion of the size of the at-risk 
population involved can be found on page 
41 and in Part III.D. 

Primary Reasons for I articipating in 
T&PD Activities. In Part 3 of the survey, 
participants were asked to indicate the 
primary reasons for their participation in 
district T&PD activities during the 1990- 
91 school year. A summary of these 
responses follows. 

Highest Positive Ratings 

72% To acquire effective instructional 
delivery skills 

69% To provide more quality 
instruction and lor services to at- 
risk pupils 

67% To learn about innovations in the 
restructuring of curriculum for 
effective teaching and learning 

Highest Negative Ratings 

31% Do not participate to learn about 
national I local models of 
continuous pupil assessments 

24% Do not participate to improve 
classroom management and 
effective recordkeeping skills 

24% Do not participate to expand their 
knowledge of academic content 
directly related to current position 



Throughout the survey, the highest posi- 
tive ratings are given consistently to those 
T&PD activities that concern instruc- 
tional practices — instructional delivery, 
services to at-risk students, restructuring 
curriculum and effective teaching and 
learning methods. 



One-third of 
the teachers 
responding 
said at least 
50 percent of 
their pupils 
were at risk of 
not complet- 
ing their 
education. 
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Seventy-two 

percent of 

trainees 

surveyed 

attended 

T&PD 

courses to 

gain effective 

instructional 

delivery 

skills. 



Conversely, 25 to 30 percent of respon- 
dents indicated that they do not partici- 
pate in T&PD activities primarily to gain 
classroom management skills, expand 
their knowledge of academic content areas 
or learn about new models of pupil 
assessment. The strongest reported 
determinant (72%) of T&PD course 
attendance concerned acquisition of 
effective instructional delivery skills. 

Impressions of LEA T&PD Programs. 

Participants were asked the degree to which 
local Chapter 2-supportedT&PD programs 
were able to provide a variety of 
instructional and professional development 
opportunities. The questions and the 
distribution of responses are summarized 
below. 

As shown, the highest Strongly Agree 
response occurred for offers opportunities 
+~ nnnifj re effective instructional practices 
while the highest Strongly Disagree was 
reported for provides training in 
administrative skills enhancement. In 
addition, if the negative responses for 
Disagree and Strongly Disagree are 



combined into a single negative rating, 
references to provides training in 
administrative skills enhancement and 
provides sufficient release time to avoid 
scheduling conflicts generated the strongest 
negative ratings at 31 percent and 18 
percent, respectively. 

Individual Course Characteristics. 
Part 2 of the survey instrument provided 
the participant with the opportunity to 
complete as m?^v as five individual course 
evaluations. A total of 1,711 individual 
course evaluation sheets were completed 
and returned. On average, each of the 975 
respondents reported taking approximately 
two T&PD courses during SY 91. 

The demographics of the T&PD courses 
taken showed considerable variety in 
content and focus. The survey instrument 
provided a list of 48 separate course titles 
from which respondents could select 
courses they had taken. However, by 
allowing respondents to write in descrip- 
tive titles for courses they had taken which 
were not listed, information was collected 
on an additional 26 T&PD courses. 



In general, your district Chapter 2-supported 
T&PD program . • • 

A*, offers sufficient opportunities for 
employee orientation, staff training 
and professional development. 

B. provides sufficient release time to 
avoid scheduling conflicts. 

C. offers opportunities to learn policies 
and procedures. 

D. offers opportunities to acquire effective 
instructional practices. 

E. provides training in noninstructional 
methods, such as enhanced classroom 
management strategies. 

F. provides training in administrative skills 
enhancement. 

G. provides training in supportive services 
skills development 

H. offers staff an adequate variety of 
training formats, such as workshops, 
seminars, conferences and structured 
course work. 



Strongly 
Agree Agree 



Strongly 
Disagree Disagree 



45% 


47% 


7% 


17o 


32% 


50% 


15% 


3% 


32% 


53% 


14% 


1% 


46% 


49% 


5% 


1% 


33% 


52% 


14% 


1% 


21% 


47% 


27% 


4% 


24% 


55% 


19% 


2% 


37% 


45% 


16% 


2% 
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Once aggregated, the data indicated that 
district trainees reported their local T&PD 
activities under 74 separate course titles. 
The titles originally listed accounted for 
only 65 percent of the courses taken by 
respondents. 

The survey revealed that the four T&PD 
courses which received the highest 
attendance percentage were Essential 
Elements of Instruction (12%), Cooperative 
Learning (10%), Writing Across Curricu- 
lum (6%) and English as a Second Lang- 
uage (6%). 

Courses in the content areas of instruc- 
tional practices are the most popular T&PD 
ac< vities in LEAs throughout Arizona. 

Participants were asked to classify the 
T&PD sessions they attended according to 
11 general content areas as well as to pro- 
vide information on the purpose, focus, 
format and location of the training sessions. 

Respondents categorized approximately 
30 percent of the 1,711 courses they 
had attended under the global content 
area of Instructional Delivery. 

The top four general content-area 
descriptions of courses identified by 
respondents were Instructional Delivery 
(30%), Academic Education (17%), Staff 
Effectiveness Training (12%) and Pupil 
Motivation (10%). 

Forty-eight percent of respondents 
identified the primary purpose of their 
T&PD courses as In-service Training* 

Definitions supplied with the survey 
describedin-service training as instruction 
delivered to staff who are in need of new 
concepts, content, strategies, or other job- 
related knowledge in a current position or 
within a current area of responsibility." 

Another 37 percent indicated Staff 
Development to be the purpose behind their 
training sessions. Staff development was 



definedin the survey as "measurable growth 
of an employee in both general knowledge 
and aptitudes that relate directly to 
performance of daily routines and assist 
individuals in relating to peers and 
supervisors. Acquisition of specific 
concepts, content, strategies, etc. is not its 
primary goal." 

Seventy percent of respondents selected 
Instructional Practices as the focus of their 
training. In regard to format, 57 percent 
reported that sessions they attended were 
considered to be workshops as opposed to 
seminars (18%), structured course work 
(17%) or conferences (9%). (See Appendix 5, 
The Local Chapter 2 Evaluation Survey, 
for these definitions of terms.) Finally, 80 
percent of all the T&PD activities were 
held on-site within the district. 

Individual Course Evaluations. 
Participants evaluated their Chapter 2- 
supportedtrainingexperiences by reporting 
their level of agreement or disagreement 
with a series of affirmative statements. 

These statements characterized the 
effectiveness of T&PD activities in terms of 
the professional and educational benefits 
derived by participants in this evaluation. 
The information below is broken out into 
three major concept areas: 

• teacher and presentation materials ef- 
fectiveness, 

• implementation of course content, and 

• survey of course outcomes. 

Two of the three concept areas contained 
specific domains which focused on different 
aspects of the participant's training 
experience. 

The complete course evaluation results are 
found in Appendix 5 and presented on the 
following pages according to major concept 
areas and associated domains. 



Area: Instructor Effectiveness 

Domain: Preparation and Knowledge 

(Questions:Part2.8Aand2.8.B) 

Inquiry: Instructor was well-prepared to 
conduct training session and was ; 
knowledgeable of the subject 
matter, 

R«sult«: Respondents gave over- 
whelmingly positive rating of 
98 percent to preparation and 
knowledge of instructors 
conducting T&PD sessions. 



represented by lower Strongly 
Agree responses and a higher 
combined negative score. 





POSITIVE RANGE 


Agree 


Strongly Agree Combined 


39.4% 


52.1% 91.5% 




NEGATIVE RANGE 


Disagree 


Strongly Disagree Combined 


6.8% 


1.7% 8.5% 





POSITIVE RANGE 


Agree 


Strongly Agree Combined 


28.5% 


69.8% 98.3% 




NEGATIVE RANGE 


Disagree 


Strongly Disagree Combined 


1.4% 


0.3% 1.7% 
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Area: 



Domain: 



Inquiry: 



Area: Presentation Materials Effec- Results: 
tiveness 

Domain: Utility of Handouts and Other 
Materials 

(Questions:Part2.8.Cand2.8.D) 



Inquiry: Handouts, audio/visual aids 
were useful in helping trainees 
understand course content and 
served a s reference tools for later 
implementation and sharing 
with peers of what was learned. 

Results: Respondents were slightly less 
positive about materials used in 
various T&PD activities, as 



Implementation of Course 
Content 

Changes in Routines 
(Questions: Part 2.9.A - 2.9.C) 

Attendance at T&PD activities 
resulted in altering instruc- 
tional, administrative or service 
delivery methods; teaching new 
content or revising existing 
policies; or improving attitude 
and commitment of students or 
staff. 

Over 15 percent of the respon- 
dents did not agree that positive 
changes occurred as a result of 
attending vari^as T&PD 
activities. 





POSITIVE RANGE 


Agree 


Strongly Agree Combined 


47.8% 


37.0% 84.8% 




NEGATIVE RANGE 


Disagree 


Strongly Disagree Combined 


12.2% 


3.0% 15.2% 
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Area: Summary of Course Outcomes 

Domain: Effects on Instruction and 
Student Outcomes 
(Questions: Part 2.10.D and 
2.10.E) 

Inquiry: 

Outcomes of attending the T&PD 
activities included improved 
abilities to teach specific content 
areas, improved motivation and 
performance of students and 
positive impact on the 
performance of at-risk students. 

Results: While responses on the impact 
of T&PD activities on teaching 
ability and students were very 
positive, respondents indicated 
that this was not true for 20 
percent of the courses taken. 

Based on the distribution of 
negative responses in this 
survey, 20 percent reflects a 
significant negative result. 



Inquiry: Outcomes of attending T&PD 
activities were increased com- 
mitment to job and increased 
understanding of educational 
and administrative models. 

Results: While 78 percent of the 
respondents agreed that T&PD 
activities had a positive impact 
on their understanding of 
educational and administra- 
tive models and caused an 
increased commitment to their 
job, over 22 percent disagreed 
with these conclusions. 





POSITIVE RANGE 


Agree 


Strongly Agree Combined 


48.0% 


29.9% 77.9% 




NEGATIVE RANGE 


Disagree 


Strongly Dim agree Combined 


18.6% 


. 3.5% 22.1% 





POSITIVE RANGE 


Agree 


Strongly Agree Combined 


48.2% 


31.9% 80.1% 




NEGATIVE RANGE 


Disagree 


Strongly Disagree Combined 


17.1% 


2.8% 19.9% 
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Area: Summary of Course Outcomes 

Domain: Commitment and Under- 
standing 

(Questions: Part 2.10.C and 
2.10.F) 



Area: 



Domain: 



Inquiry: 



Results: 



S ummar y of Course Outcomes 
Relevance 

(Question: Part 2.10.A) 

The course content exceeded my 
expectations in terms of quality , 
relevance to daily routines and 
attention to professional needs. 

While 77 percent of the 
respondents agreed with the 
statement on relevance of T&PD 
activities to daily routines and 
professional needs, 23 percent 
replied that they did not agree. 
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Improving the 
quality of 
instruction 
and 

enhancing 
services to at- 
risk students 
motivated 
nearly 70 
percent of the 
respondents 
to participate 
in T&PD 
programs 

40 





POSITIVE RANGE 


Agree 


Strongly Agree Combined 


48.2% 


29 1% 77.3% 




NEGATIVE RANGE 


Disagree 


Strongly Dit agree Combined 


20.0% 


2.8% 22.8% 



♦ Comments 



One of the most striking aspects of the 
responses aggregated across participants 
in T&PD activities at the LEA level is the 
heavy emphasis placed on instructional 
activities and training. Seventy-two per- 
cent of the participants indicated that 
their primary reason for participating in 
T&PD activities was to acquire more 
effective instructional delivery skills while 

69 percent cited providing more quality 
instruction and/or sez'vices to at-risk 
students. 

In addition, the single most-often-attended 
course was Essential Elements of 
Instruction, which accounted for 12 percent 
of the 1,711 courses taken by respondents 
during SY 91. This is significant in that 74 
different courses were reported from 43 
districts which operated independently in 
terms of their T&PD strategies, focus and 
course activities. 

Thirty percent of respondents chose 
instructional delivery as the primary content 
area heading under which they would 
classify the sessions they attended, while 

70 percent said that instructional practices 
constituted the primary course focus. 

An analysis of the statistics collected on 
individual course evaluations revealed a 
declining trend in the satisfaction 
participants experienced with the training 



sessions they attended and with the 
eventual impact this training had on their 
students or job activities. 

This trend is clearly seen in the combined 
negative ratings given to each of the six 
course-evaluation domains. As reported 
below in Individual Course Evaluations: 
Concept Areas and Domains, it is evident 
that participants' negative responses to 
statemer i about the courses they attended 
increased as the domains increasingly 
focused on outcome effects. 



Individual Course Evaluations 
Concept Areas and Domains 


Concept Area/Domain 


Combined 
Negative 
Response 


Instructor/Preparation 
and Knowledge 


2% 


Instructor/Handout 
Materials 


9% 


Implementation/Changes 
in Activities 


15% 


Outcomes/Effects on 
Instruction and Students 


20% 


Outcomes/Commitment 
and Understanding 


22% 


Outcomes/Relevance 


23% 



The vast majority of participants agreed 
that T&PD instructors generally were well- 
prepared, knowledgeable and provided 
useful information. A large proportion ( 85%) 
of respondents indicated that they made 
efforts to implement the skills and 
knowledge gainedin their t raining sessions 
either in their daily work routines or in the 
office and classroom. 

However, 20 percent or more felt that their 
possession of these newly acquired skills or 
knowledge did not significantly impact their 
job performance or outcomes for their 
students. 



There may be a variety of reasons why this 
declining trend from T&PD participation 
to eventual impact exists* First, it may be 
difficult for the classroom teacher to 
significantly change methodologies, content 
or practices over the short run. If T&PD 
courses are attended while classes are in 
session, teachers may not have the time or 
flexibility to change activities immediately 
after receiving training. 

Second, the drop in impact may be due to 
the evaluation and follow-up processes 
undertaken by the districts. If teachers or 
administrators attend a variety of T&PD 
activities but are not required to report 
acquired content or implementation plans 
then these participants may lack incentive 
to act on the knowledge or skills they 
received in their Training & Professional 
Development classes. 

Potential Impact on At- Risk Students. 
Fifty-seven percent (804) of the evaluation 
survey responses received were from 
classroom teachers. Approximately 33 
percent or 265 teachers indicated that at 
least 50 percent of their students are 
considered at risk of not completing their 
high school education. 

With an average class size reported as 25, 
approximately 3,312 at-risk students (265 
teachers averaging 25 students per class 
with 50 percent at risk) may be affected by 
Chapter 2-funded T&PD activities 
evaluated in this study. Extrapolated to 
the total population of teachers (5,139) 
across all 43 districts in the evaluation 
study group, and using the same statistical 
averages, approximately 21,200 at-risk 
students would have been impacted by 
Chapter 2-funded T&PD activities in 
Arizona during SY 91. 

The calculations cited above are based on 
the conservative assumption that all 
teachers surveyed in the evaluation, as 
well as those contained in the universe of 
T&PD participants, teach only one class of 
25 students per day. Additional informa- 



tion obtained from the survey indicated 
that approximately oo percent of the 
teachers responding were teaching at the 
secondary level (grades 7-12). Using one 
class per day of 25 students as the basis for 
calculating total student coverage results 
in a very conservative estimate. 

. More realistic estimates may be obtained 
by breaking down the total count of teachers 
reporting 50 percent or more at-risk 
students in their classrooms into elementary 
and secondary categories and then 
increasing the number of .classes taught 
per day at the secondary level. 

This analysis again draws on the statistics 
derived from 804 teacher responses obtained 
in the evaluation. The statistics and 
assumptions used are outlined below. 

• Thirty-three percent of teachers report 
having at least . 50 percent at-risk stu- 
dents in their classrooms. 

• Fifty-five percent of teachers report teach- 
ing at the 7-12 grade level, leaving 45 
percent teaching in Preschool through 
grade 6. 

• The total population of teachers attend- 
ing Chapter 2-funded T&PD activities 
was 5,139. 

• Teachers at the secondary level teach up 
to three classes per day. 

Using these statistics, the potential impact 
of T&PD activities may be assessed for two 
scenarios: secondary teachers teaching at 
first two and then three classes per day. 
Assuming secondary teachers teach two 
classes per day, the number of at-risk 
students potentially impacted is calculated 
to be 32,863 or 6.8 percent of the students in 
the 43 districts in this study. The figures 
used in these calculations are as follows: 

• 5,139 teachers, 33 percent of whom teach 
classes having greater than 50 percent 
at-risk students = 1,696 teachers. 
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just two 
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the number of 
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41 



Based upon 
conservative 
estimates, 
researchers 
believe that 
nearly 45,000 
at-risk 
students 
benefited in 
SY91 and 
beyond from 
being taught 
by teachers 
participating 
in Chapter 2- 
funded T&PD 
programs. 

ERIC 42 



• 45 percent of 1,696 teachers, at the 
elementary level, teaching one class per 
day averaging 25 students per class, 50 
percent of whom are at risk = 9,538 
at-risk students. 

• 55 percent of 1,696 teachers, at the 
secondary level, teaching two classes per 
day averaging 25 students per class, 50 
percent of whom are at risk = 23,325 
at-risk students. 

• Then, 9,538 students at the elementary 
level + 23,325 at the secondary level = 
32,863 at-risk students. 

Using the same calculations, but as- 
suming secondary teachers teach three 
classes per day results in 34,988 at-risk 
students potentially impacted at the 
secondary level, plus 9,538 elementary 
students, for a total at-risk student 
population of 44,526. 

As with the original calculation of 21,200 
at-risk students based on one class taught 
per day by all teachers, the higher estimates 
derived above also may be considered 
conservative. This is because the definition 
of an at-risk classroom is based on having 
at least 50 percent at-risk students enrolled. 

From the survey responses, 1,696 (33%) of 
5,139 teachers fell into this category. This 
leaves 3,443 teachers estimated not to have 
taught classes with 50 percent or more at- 
risk students but who may have had, in 
their opinion, anywhere from 0-49 percent 
at-risk students in the classroom. 

The degree of underestimation is impos- 
sible to determine from the available 
data. However, these factors do suggest 
that the figures derived above understate 
the actual number of at-risk students 
potentially affected by Chapter 2-funded 
T&PD activities. 

Total Student Coverage. Using the 
methodology applied earlier for calcu- 
lating the numbers of at-risk students, 



estimates of general student population 
coverage may be made for the purpose of 
investigating the potential impact on stu- 
dents of Chapter 2-funded T&PD activi- 
ties. 

Direct survey responses were received from 
975 individuals, of whom 804 were 
classroom teachers. The coverage of 
students assigned to this sample group of 
teachers is estimated to be as high as 
20,100 students, based on the reported 
average class size of 25, if each teacher has 
one class per day. Extrapolated to the 
5,139 teachers in the study group, student 
coverage is estimated at 128,475 pupils. 

Similar to the at-risk student calculations, 
this estimate of total student coverage is 
conservative in that teachers at the 
secondary level are assigned more than one 
class per day. As noted above, survey 
responses indicated approximately 45 
percent of study-group teachers teach at 
the elementary level with 55 percent at the 
secondary level. 

Both lower and upper bounds of the number 
of students potentially impacted may be 
calculated by splitting the 5,139 teachers 
into suggested elementary and secondary 
teacher proportions of 45 and 55 percent, 
respectively, while assuming the assign- 
ment of teaching load to be between one 
and three classes per day. 

Using these revised conditions and the 
average class size of 25, it is estimated that 
the total number of students taught by 
teachers who participated in T&PD 
activities during SY 91 is between 128,000 
and 270,000 pupils. 

Variety of T&PD Activities. 
Responses to survey instruments 
constructed for this evaluation accounted 
for 1,711 individual courses attended 
during SY 91. This implies upwards of 
9,400 Chapter 2-supported T&PD activi- 
ties attended by individuals employed in 
the 43 districts included in this study and 



occupying one of the three job categories 
studied (approximately 1.61 courses per 
individual covering 5,819 trainees). 

Survey respondents reported 74 different 
descriptive course titles, including 48 
provided by the ADE in a listing supplied 
with the survey form and 26 additional 
titles self-reported by the respondents. 
Given that over three-fourths of respon- 
dents were classroom teachers who are 
concerned primarily with instruction and 
learning, the fact that 74 separate 
descriptive titles were reported represents 
an unexpectedly wide variety of T&PD 
courses offered by the LEAs. 

Actual Impact on Students in the 
Classroom. In this evaluation, partici- 
pants in T&PD activities reported very 
positive assessments of their instructors 
and materials for the courses they 
attended. A very large proportion also 
gave positive responses to queries on 
whether they were utilizing the newly 
acquired knowledge and skills by 
incorporating them into their daily office 
routines or classroom activities. 



negative responses through four domains 
of survey questions. 

The statistics reveal that approximately 
20 percent of the T&PD trainees dis- 
agreed with positive statements about the 
impact of their training on student 
motivation or performance. This 
increasingly negative response rate 
observed as the domain of questions moved 
from participation to implementation and 
finally to the impact on students may 
reflect difficulties teachers experienced in 
implementing new knowledge and skills 
acquired in T&PD sessions. 

The short time lines and the heavy 
workloads characteristic of the academic 
year may not provide teachers with 
sufficient flexibility to significantly change 
the content or instructional procedures they 
already use. This suggests that a longer 
term analysis of student attitudes, 
expectations and performance may be 
required to determine the degree to which 
teacher involvement in T&PD activities 
culminates in measurable changes in 
student outcomes. 



Domain 


Combined 
Negative 
Responses 


Instructor Knowledge/ 
Preparedness 


2% 


Materials and Handouts 


9% 


Implementation of 
Skills/Knowledge 


15% 


Outcomes from 
Implementation 


20% 



However, when teachers were questioned 
about the eventual impact that their 
attendance at these T&PD activities had 
on students in their classrooms, the 
percentage of combined negative responses 
increased. The table above shows this 
progression toward increasingly larger 



ERLC 




When teach- 
ers return to 
the classroom 
with new 
skills they 
can immedi- 
ately imple- 
ment, benefits 
to students 
tend to 
increase. 



43 



be:: 



4. Survey of District T&PD Coordinators 



To conduct the second half of the 
Evaluation of Chapter 2 Program 
Effectiveness, Research and Development 
Division researchers solicited information 
directly from each of the 43 study-group 
districts, previously defined in Part I.C, by 
inviting responses from district T&PD 
coordinators participating at the local level. 

Data were collected on the operation of 
local educational agency T&PD programs 
from the perspective of the district person- 
nel responsible for directing these pro- 
grams. In many cases this person was the 
district's designated Chapter 2 program 
coordinator. 

Therefore, the Chapter 2 Coordinator 
Questionnaire was utilized in concert with 
the Local C 2 Evaluation Survey, 

described previously in II.B.3, to gain a 
more comprehensive picture of the relative 
effectiveness of the Arizona Chapter 2 
Program. 



• Focus 

The Chapter 2 Coordinator Questionnaire 
was designed by the ADE to investigate 
the administrative and organizational 
structure of Chapter 2-supported T&PD 
activities within the study-group districts. 
This involved the collection and analysis of 
general demographic data for district T&PD 
coordinators as well as: 

• Identifying the primary focus of each 
Chapter 2-supported district T&PD 
program, 

• Identifying district use and structure of 
committee review teams in making policy 
decisions and setting program guidelines, 

• Documenting the district process for 
approving staff T&PD activities, 



• Documenting the district training 
evaluation and planning processes, 

• Collecting district T&PD coordinators 
impressions of the Arizona Chapter 2 
Program, and 

• Evaluating LEA/SEA interactions. 



♦ Methodology 

To prepare for designing the survey 
instrument, ADE researchers conducted 
in-person and telephone interviews with 1 1 
district T&PD coordinators. Interview 
discussions centered on activities at the 
district level which supported T&PD 
programs. Interactions of District Chapter 
2 Programs of Training and Professional 
Development, Diagram 1, illustrates 
possible interactions among district T&PD 
coordinators andthe state Chapter 2 Office, 
district committee review teams, district 
policies and goals, and the LEA staff 
participating in T&PD activities. 

From the initial discussions, the existence 
of two separate application/allocation 
processes became evident. First, the district 
Chapter 2 office followed an application/ 
allocation process with the state office. This 
allocation process was composed of the 
procedures, guidelines and restrictions with 
which districts must comply to receive 
funding for proposed Chapter 2 programs, 
including T&PD. 

Second, staff within the LEA wishing to 
participate in available district T&PD 
activities usually followed some type of 
district application process to receive 
permission and funding to attend T&PD 
sessions. This process might include 
requirements for participants tc provide 
formal evaluations of the T&PD sessions 
after attendance. The district office also 



might undertake its own evaluation and 
planning process to document the outcomes 
of T&PD activities from year to year or to 
develop long-range strategic training plans. 

In addition to the staff application/training 
allocation process, some districts utilized 
oversight committees or committee review 
teama to administer their Chapter 2 T&PD 
program. Members of these teams included 
administrative staff from the district, 
instructional staff, parents and rep- 
resentatives of private schools within the 
district that share in Chapter 2-funded 
activities. 

Finally, LEA Chapter 2-supported T&PD 
programs are influenced by goals and 
objectives set forth by local governing 
boards and district administrators. 

Obtaining this specific information 
through the preliminary interviews en- 
abled ADE researchers to create a survey 
instrument in seven parts, composed of 
questions on each meg or process identified 
above. This instrument, the Chapter 2 
Coordinators Questionnaire, was sent to 
district T&PD coordinators in the 43 
districts participating in the study group. 
Surveys were returned by 41 of these 43 
individuals. This resulted in a survey 
response rate of 95 percent. 



♦ Findings 

Respondent Demographics. Part 1 of 
the survey provided demographic data on 
the 41 respondents. District T&PD 
coordinators were found to spend an average 
of four years administering staff 
development programs; the longest time 
reported was 10 years. 

Ninety-three percent of the respondents 
(38 of 41) reported that coordination of 
Chapter 2 T&PD programs was not their 
primary job responsibility. The other three 



districts employing full-time Chapter 2 
coordinators were dissimilar in respect to 
student population. Enrollment figures for 
these districts were 8,917 and 747 and 
60,556 respectively. 

All 41 districts indicated they had 
coordinators who had been classroom 
teachers. The average length of time spent 
as a classroom teacher was 10 years while 
21 years was reported as the longest time 
period. 

Use of Committee Review Teams. In 
Part 2 of the survey, 83 percent (34 of 41) of 
the coordinators indicated that their 
districts utilized committee review teams 
or committee structures to oversee Chapter 
2-funded T&PD activities. Only seven 
districts reported not using any type of 
committee structure. 

As shown in Chart 3, Use of Committee 
Review Teams in Chapter 2-funded T&PD 
Activities, 34 of the 41 districts 
responding (83%) used committee struc- 
tures to develop general policies, proce- 
dures and/or long-range strategic plans 
while 31 districts (76%) used committees 
to determine the type of T&PD courses 
offered In addition, 24 districts (59%) used 
committees to review applications for staff 
training and 13 districts (32%) used 
committees to set limits on the use of funds. 

Chart 4, District Review Team Member- 
ship: Categories Greater Than 50 Percent, 
shows that 94 percent of the responding 
districts (32 of 34) used committee 
structures to oversee Chapter 2-funded 
T&PD activities that included teachers. 
Seventy-seven percent (26 of 34) indicated 
inclusion of district superintendents while 
79 percent (27 of 34) included principals 
and 65 percent (22 of 34) involved other 
district personnel. Finally, 66 percent 
(23 of 34) indicated that their committee 
review teams included parents. The 
committee review teams were found to 
average 14 members. 



Chart 3 

Use of Committee Review Teams in Chapter 2-Funded 
Training & Professional Development Activities 




Policies and 
Strategic 
Planning 



76% 
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Source: Arizona Department of Education Chapter 2 District Coordinator Questionnaire: School Year 1991. 
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Chart 4 

District Committee Review Team Membership 
Categories Greater Than 50 Percent 



94% 



II 

it 

""liiil 



Mil*' 



77% 



79% 



+ 



11 

a 

!!!;!!! 
Pi 
iiii 

•ft!:! 

ill 
SIS 

llflll! 
iiii; 

■f:U . . 
: 1 33? 1 1 1 ■ 



.jjiijjjpjj|j 



iiiiiiiiiiiii 

111! 

ill! 



:::i:!|t::: : : 



+ 



'ill 

111 



iilii 

iiii 



65% 



68% 



;:::•!•:•:•:: 



iiiiiiii 

iilii 

m 

i!iiiiii : 

WW 

HI 

MM- 



iiiiiii 



Teachers 



Superintendents Principals 



Other District 
Personnel 



Parents 



Source: Arizona Department of Education Chapter 2 District Coordinator Questionnaire: School Year 1991. 
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Focus of District T&PD Programs* 

District coordinators were asked in Part 2 
of the survey to identify the primary focus 
of their Chapter 2-supported T&PD pro- 
grams. Of 41 surveys returned, 8 yielded 
unusable data or gave no response* As 
shown below, 88 percent of the 33 
responding district T&PD coordinators 
selected either Improved Instructional 
Delivery (42%) or Increased Staff 
Effectiveness (46%) as the primary emphasis 
of training. 



Primary Focus of Chapter 2-Funded 
Training & Professional 
Development Programs 



T&PD Coordinators 

Reporting 
Enhancements to: 


Number 
of 

Districts 


Percent 


Instructional 
Delivery 


14 


42% 


Educational Content 


0 




Student Motivation 


1 


3% 


Assessments 


0 




Staff Effectiveness 


15 


46% 


Instructional 
Leadership 


0 




Teacher Coaching 


0 




Curriculum 


1 


3% 


Schoolwide 
Improvements 


2 


6% 


School Environment 


0 




Parental 
Involvement 


0 




Usable 

Nonduplicated 
Responses 


33 


100% 



This dual focus on Improved Instructional 
Delivery and Increased Staff Effectiveness 
confirms findings attributable to LEA 
trainees whose responses to the Local 
Chapter 2 Evaluation Survey heavily 
emphasized the importance of instruction- 
related T&PD activities as a determi n a n t 
of course attendance. 



Variety of T&PD Delivery Methods. 
To understand whether such T&PD 
activities were aligned with district- 
supported goals and objectives, district 
T&PD coordinators were asked to express, 
as a proportion between 0 and 100 percent, 
the emphasis their Chapter 2-funded T&PD 
program placed on the following methods 
by which Chapter 2-supported T&PD 
activities were made available to district 
personnel. 

• Minigrants for Innovative Staff- 
Initiated Training. During the inter- 
views held prior to survey development, 
a number of local T&PD coordinators 
pointed to their use of minigrants to 
support innovative or new approaches to 
staff development. These minigrants 
differ from other T&PD activities in 
that they are not directly aligned to 
existing district in-service training. 
Rather, the minigrants permit staff 
members to propose training services 
based upon individual need. 

• District-Sponsored / Supported In- 
services. These more traditional T&PD 
offerings refer to in-services which 
initially are selected by the district. The 
implication is that staff choose from a 
menu of T&PD activities which are 
officially supported or sponsored by the 
district, based on specific, predetermined 
goals and objectives. 

• Staff-Requested In-services Not 
Sponsored by the District These T&PD 
activities are identified by the individual 
and not specifically sponsored by the 
district. Under this option, district 
policies may allow staff to attend a wider 
array of in-service options, including 
those which are specifically tailored to 
the needs of the individual teacher or 
administrator. 

• Other. ThisoptionalloweddistrictT&PD 
coordinators to react 6n a case-by-case 
basis regarding the potential for 



sponsorship and support of selected 
T&PD activities. 

Primary T&PD Delivery Method. The 

Chapter 2 Coordinator Questionnaire 
instructed district T&PD coordinators to 
determine which delivery methods were 
emphasized in their Chapter 2- supported 
training program. 

Also they were asked to express the degree 
of importance on each method identified as 
a proportion of total emphasis. Proportions 
suggested by coordinators could not exceed 
100 percent. The table reflects the 
reclassification of Other responses under 
the appropriate heading or under "emphasis 
unclear." 



Methods of District 
Sponsorship 


Number 
of 

Districts 


Percent 


Staff-Initiated 
Mini grants 


1 


3% 


District-mandated 
In-services 


14 


34% 


Staff-requested 
In-services 


19 


46% 


Emphasis Unclear 


4 


10% 


Unusable or 
Missing Response 


3 


7% 


Total Districts 
Responding 


~4l 


100% 



The previous data box also indicates that 
14 of 41 district T&PD programs (34%) 
emphasized district-supported or sponsored 
T&PD activities over staff-initiated in- 
services. 

However, 19 districts (46%) responded that 
most T&PD activities resulted from 
personal requests based on individual 
needs. 
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By including the mini grants category with 
staff-initiated requests, the proportion 
increased to 49 percent, suggesting that 
about half of the study-group districts 
emphasized a strong measure of staff 
participation in the selection of Chapter 2- 
funded T&PD activities. 

Therefore, LEAs were split on the type of 
T&PD delivery methods that were 
emphasized in their training program. 

District Process for Approving Staff 
Training. In Part 3 of the survey , a num- 
ber of questions were asked of T&PD 
coordinators about staff participation in 
the T&PD application/allocation process. 

Included were questions concerning the 
application, review and decision-making 
processes used by districts to monitor 
participation of staff in T&PD activities; 
whether any formal limit was set on the 
amount of fuxidlug available to train 
applicants; and whether any district 
personnel were required to attend specific 
staff development in-services. The data 
collected in these areas are summarized on 
the next page. 




Access to 
skills training 
for classroom 
teachers with 
expectations 
of immediate 
applicability 
gets high 
marks from 
trainees. 
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Summary of Responses for 
District Process Used to Approve 
Staff Training 

• 66 percent of districts do not require a 
formal application for Chapter 2-funded 
T&PD activities. 

• 64 percent have no formal review process 
for approving requests for T&PD 
activities. 

• When asked who made the final decision 
to approve staff training: 

36 percent indicated Chapter 2 
Coordinator or Staff Developer 
22 percent indicated district admin- 
istrators, and 

20 percent indicated school principals 
or local administrators. 

• 95 percent of the districts indicated that 
no formal limit is imnosed on the dollar 
amount any one person may expend for 
training. 

• 71 percent of districts required certain 
staff members to attend specific Chapter 
2-funded T&PD activities. Of these 
districts, 89 percent required attendance 
by new teachers while 59 percent 
required attendance by new principals 
or assistant principals. 

• 27 percent of districts required annual 
training for instructional personnel in 
specific content areas, including 
Thinking Skills, Essential Elements of 
Instruction, Science, Adaptive 
Curriculum, Language Arts, Chemical 
Abuse Prevention and Effective Schools. 



This detailed survey information provides 
an insightful look at the structure of the 
application and approval process for staff 
training within Arizona school districts. 
First, 66 percent (27 of 41 districts) reported 
that formal applications were not required 
whenever staff attended T&PD activities. 



Additionally, 64 percent (25 of 39 dis- 
tricts) indicated no formal review process 
being used for approval of T&PD requests. 
These findings seem contrary to a 
response in Part 2 of the survey which 
revealed that 59 percent (24 of 41 districts) 
use a committee structure to review train- 
ing applications. Excluding the six dis- 
tricts which reported no use of committee 
structures increases the proportion of 
districts using such committees to review 
applications to 69 percent (24 of 35 
responding districts). 

Note, however, that terminology used in 
questions about the approval/review of 
training requests included an emphasis on 
a formal process. The survey results 
indicated that while training applications 
and their subsequent review are required 
in some districts, most do not have formal 
policies and procedures for approving staff 
training. 

Seventy-one percent of district T&PD 
coordinators indicated that personnel in 
specific job categories are required to take 
courses in training and professional 
development. Most of these staff members 
were teachers new to the district (89%) or 
newly hired administrators (59%). 

Eleven districts, comprising 27 percent of 
respondents, required annual training for 
instructional personnel in specific course 
content areas including Thinking Skills, 
Essential Elements ofInstruction t Science, 
Adaptive Curriculum, Language Arts, and 
Effective Schools. This requirement is 
additional evidence of the heavy emphasis 
on instruction-related activities within 
district T&PD programs that the survey 
revealed. 

District T&PD coordinators were asked to 
identify who makes the final decision to 
fund staff requests for T&PD activities. 
Their responses are itemized on the next 
page. 



Final Decision Authority 
Staff Applications for Training 


Category Districts Percent 

District 
Administration 

Superintendent 9 
Administrator* 1 
Academic Services -1 

Subtotal U 22% 


District Stall Support 

fifcnff Develooer 
Chapter 2 Coordinator 
Subtotal 


5 

12 

18 


36% 


Local Administration 

Principal 

Administrator* 

Supervisor* 

Subtotal 


8 
1 

— i 

10 


20% 


Chapter 2 Committee 
Subtotal 


_a 

9 


18% 


Other 

^cVy*] Board 
District Council 
Subtotal 


1 

2 


4% 


Total Usable Items 


50** 


1C0% 


No RsspozxM or Umasablsx 3. 






• These job title* were identified from response* 
provided under Other on the survey form. 


This count allows for multiple responses by 

districts. 



Fourteen of the 41 district T&PD 
coordinators responding to the survey 
provided multiple answers to the question 
examined in the data table above, 
suggesting that decisions to approve staff 
tr aini ng applications are not centralized in 
all districts. 

Personnel cited most often (18 of 50 
responses) as being the primary decision 
authority were those individuals who serve 
as district T&PD coordinators. In the stu dy- 
group districts, a T&PD Coordinator is 
either the district's designated Chapter 2 
Program Coordinator a staff developer who 
manages T&PD activities. Following the 
District Staff Support category with 18 
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responses came District Administration 
(11 responses), Local Administration 
(10 responses) and the Chapter 2 Review 
Committee (9 responses). 



Summary of Responses on 
District T&PD Evaluation and 
Planning Processes 

• 66 percent (27 of 41) study-group districts 
conducted formal evaluations of T&PD 
activities. 

• 68 percent (28 of 41) indicated that all 
participants in T&PD activities were 
required to complete evaluations. 

• 37 percent (15 of 41) indicated that 
evaluations were obtained through 
feedback from staff meetings and peer 
discussions. 

[Note: The percentage* cited above allow for 
selection of multiple responses.] 

• 66percent(27of41)indicatedthatT&PD 
evaluations were submitted to district 
T&PD coordinators; 17 percent (7 of 41) 
indicated that evaluations were given to 
district administration and 46 percent 
(19 of 41) indicated that evaluations are 
sent to the in-service providers. 

• 95 percent (39 of 41) reported thai 
evaluations were used to provide 
feedback on specific training delivered. 

• 68 percent (28 of 41) reported evaluations 
were utilized for strategic planning of 
future T&PD activities. 

• 79 percent (32 of 41) indicated that 
training effectiveness reports were 
delivered to the Chapter 2 Coordinator; 
73 percent (30 of 41) indicated delivery 
to the StafFDeveloper and61 percent (25 
of 41) indicated these reports were 
delivered to the District Superintendent. 



District Evaluations of T&PD 
Activities. In Part 4 of the survey, district 
T&PD coordinators were asked to provide 
information on processes they employ to 
evaluate T&PD activities. These questions 



included whether or not the district had a 
formal evaluation process, the primary 
methods used to conduct the evaluations, 
to whom they were submitted and how the 
evaluations were utilized in terms of the 
administration of district T&PD programs. 
The statistics collected from this part of the 
Chapter 2 Coordinator Questionnaire are 
presented on page 51, 

As shown on the previous page, 66 percent 
(27 of 41 districts) revealed that they 
conducted formal evaluations of T&PD 
sessions. This means f hat approximately 
34 percent (14 study-group districts) do not 
conduct formal T&PD evaluations. 
However, district T&PD coordinators also 
were asked how evaluations, formal or not, 
were conducted. Over 68 percent (28 of 41 
districts) responded that all individuals 
attending T&PD activities were required 
to complete some type of training or course 
evaluation. 

The distinction between formal and 
informal evaluations at first may appear to 
be significant. That is, with 34 percent of 
districts failing to conduct formal 
evaluations, questions might arise as to the 
adequacy of their accountability systems. 
There is an underlying assumption that 
districts which conduct formal evaluations 
do so under specific policies and procedures 
enabling them to construct valid indicators 
from which to ascertain the effectiveness, 
impact and value of their T&PD programs. 
However, closer examination of the 
responses reduces this concern. 

Table 3, Selected Information on the Use of 
Application and Evaluation Procedures in 
T&PD Activities , itemizes the responses of 
41 districts on questions concerning their 
use of formal application and evaluation 
procedures and on the primary functions of 
committee review teams. The information 
presented in the table indicates a higher 
degree of monitoring of Chapter 2 -funded 
T&PD activities than was previously 
suggested. While 14 districts reported not 
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using a formal evaluation process, four of 
these districts required every participant 
attending T&PD activities to complete 
some type of course evaluation. 

Of the 10 remaining districts, five 
required either formal applications 
before attendance and/or conducted a 
formal review of applications prior to 
approval of T&PD requests. This means 
only five districts ( 12%) reported no formal 
application or evaluation procedures, no 
formal xeview of applications submitted 
and no requirement that participants 
provide evaluations of their training 
experiences. Of these five districts, three 
contained approximately 2,000 or fewer 
students, one contained 9,200 students and 
only one district had over 15,000 students. 

Ninety-five percent (39 of 41 districts) indi- 
cated that evaluations of some type were 
used to provide feedback on T&PD ses- 
sions. When combined with the statistics 
on formal evaluations, these responses 
demonstrated that a majority of the dis- 
tricts do utilize some type of evaluation 
system to make T&PD planning decisions — 
if not in a formal manner then through 
informal methods such as staff meetings, 
peer discussions and information sharing. 
In addition, over 68 percent (28 of 41) dis- 
tricts stated that these evaluations were 
used for strategic planning of future T&PD 
activities. 

In summary, although 14 of 41 districts 
responding to the survey do not utilize 
formal application procedures, nearly all of 
the districts undertake some type of 
application review and/or training 
evaluation process for Chapter 2- 
supported T&PD activities. Thus, further 
investigation into the adequacy of 
accountability systems used in districts 
not employing formal post-attendance 
evaluations of T&PD activities may be 
warranted in only a small number of the 
43 districts contained in the study group. 



Table 3 

Use of Application and Evaluation Procedures in T&PD Activities 



District SY 91 
Record Student 
Number Enrollment 



ERIC 



Formal Application 

AND 

Application Review 



Require 
Formal 
Application 



Conduct 
Formal 
Review 



Formal 
Evaluation 



Require Require 
Formal Participant 
Et mixtion Evaluation 



1 


747 


No 


No 


Yes 


No 


2 


936 


No 


No 


No 


No 


3 


1,007 


No 


No 


Yes 


Yes 


4 


1,521 


Yes 


Yes 


No 


No 


5 


1,561 


No 


No 


No 


No 


6 


2,020 


No 


No 


No 


No 


7 


2,579 


No 


Yes 


No 


No 


8 


2,603 


No 


No 


No 


Yes 


9 


2,960 


No 


No 


Yes 


Yes 


10 


4,078 


Yes 


No 


No 


Yes 


11 


4,364 


No 


Yes 


Yes 


Yes 


12 


4,386 


Yes 


Yes 


Yes 


Yes 


13 


4,494 


No 


Yes 


Yes 


No 


14 


5,309 


No 


No 


No 


Yes 


15 


5,381 


Yes 


No 


Yes 


Yes 


16 


5,568 


Yes 


Yes 


Yes 


Yes 


17 


5,720 


No ' 


• No 


Yes 


Yes 


18 


6,608 


No 


No 


Yes 


Yes 


19 


7,065 


No 


No 


Yes 


Yes 


20 


8,509 


No 


No 


Yes 


No 


21 


8,720 


Yes 


Yes 


Yes 


Yes 


22 


8,916 


Yes 


No 


Yes 


Yes 


23 


9,203 


No 


No 


No 


No 


24 


9,643 


ISO 


INO 




Yes 


25 


11,056 


No 


No 


Yes 


Yes 


26 


11,219 


No 


No 


Yes 


Yes 


27 


11,228 


Yes 


No 


No 


No 


28 


11,497 


Yes 


No 


Yes 


Yes 


29 


12,010 


Yes 


Yes 


Yes 


Yes 


30 


12,526 


Yes 


Yes 


No 


No 


31 


13,162 


No 


No 


Yes 


Yes 


32 


14,210 


No 


No 


No 


Yes 


33 


15,080 


No 


No 


No 


No 


34 


16,833 


No 


Yes 


Yes 


Yes 


35 


21,076 


Yes 


Yes 


Yes 


Yes 


36 


21,495 


No 


N/R 


Yes 


Yes 


37 


21,900 


No 


Yes 


Yes 


Yes 


38 


23,894 


No 


N/R 


Yes 


Yes 


39 


28,657 


Yes 


Yes 


No 


No 


40 


60,556 


No 


No 


Yes 


Yes 


41 


67,695 


Yes 


Yes 


Yes 


Yes 




Total Yes: 


14 


14 


27 


28 




Total No: 


27 


25 


14 


13 




No Response: 




2 








Percent Yes: 


34% 


36% 


66% 


68% 




Percent No: 


66% 


64% 


34% 


32% 



None 



Use of Committee 
to Review 
District T&PD Activities 

Determine Review Determine 
Policies/ Application T&PD 
Planning Courses 



✓ 
✓ 



✓ 
✓ 



97% 
3% 



Note: Percent calculations exclude "No Response" values. 

Source: Arizonn Department of Education Chapter 2 District Cooniinator Questionnaire, School Year 1991. 



69% 
31% 



✓ 
✓ 





✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 




✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 




✓ 


✓ 




✓ 


- ✓ 


✓ 


✓ 


✓ 


✓ 




✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 




✓ 


✓ 






✓ 






✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


✓ 




✓ 


✓ 


✓ 


✓ 


✓ 




✓ 


✓ 


✓ 


✓ 


✓ 






✓ 




✓ 


✓ 


✓ 




✓ 


✓ 


✓ 


✓ 


✓ 


✓ 


34 


24 


31 


1 


11 


4 



11% 



60 
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District T&PD coordinators were queried 
as to the recipients of T&PD course 
evaluations. Responses to this question 
revealed that in 66 percent (27 of 41 dis- 
tricts) the course evaluations went to the 
district T&PD coordinator and in 37 per- 
cent (15 of 41) it was district administra- 
tion that received them for review. Forty- 
six percent (19 of 41 districts) indicated 
that the local trainer or in-service provider 
also received staff evaluations of T&PD 
courses attended. A summary of these 
responses is provided below. 



Recipients of T&PD Evaluations 

(Based on Multiple Responses from 41 



Summary of Chapter 2 Coordinator 
Questionnaire Results 



Area A: Impressions of 

Arizona's Chapter 2 
Program 

Domain: Variety of Available Training 
(Survey Questions: Part 5.1, 5.2) 

Inquiry; Statements focus on the use of 
Chapter 2-funded programs to 
provide a wider variety of T&PD 
activities than otherwise would 
be possible. 



Surveys) 




















POSITIVE RANGE 


Category 


Districts 


Percent 














Agree 


Strongly Agree Combined 


School Principal 


15 


37% 




24.0% 


74.0% 98.0% 


Chapter 2 










NEGATIVE 2ANGL 


Coordinator 


27 


66% 
















Disagree 


Strongly Dig agree Combined 


Review Committee 


8 


20% 




2.0% 


0.0% 2.0% 


In-house Trainer or 












Outside Consultant 


19 


46% 






District 










Administration 


7 


17% 






School Board 


2 


5% 




At-Risk Student Impact 






Domain: 


Other 


3 


7% 




(Survey Question: Part 5.3) 



Impressions of Arizona's Chapter 2 
Program and LEA/SEA Interactions. 

Parts 5 and 6 of the survey asked district 
T&PD coordinators to respond to a series of 
affirmative statements on both the general 
usefulness of the T&PD program and the 
services provided by the state Chapter 2 
Office. The information presented at the 
beginning of the next column summarizes 
the responses for two mggor concept areas 
that researchers identified. 



Inquiry; Statement on the positive impact 
that T&PD activities have on at- 
risk students 





POSITIVE RANGE 


Agree 


Strongly Agree Combined 


45.0% 


52.0% 97.0% 




NEGATIVE RANGE 


Disagree 


Strongly Di»agree Combined 


3.0% 


0.0% 3.0% 
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Area B: LEA/SEA Interactions 



Domain: Local Innovation 

(Survey Question: Part 6.1) 

Inquiry: Statement on the usefulness of 
ADS application and guideline 
materials to promote innovation 
in the design and imple- 
mentation of educational 
projects and activities. 



POSITIVE RANGE 

Agree Strongly Agree Combined 

58.0% 42.0% 100.0% 

NEGATIVE RANGE 

Ditagree Strongly Disagree Combined 

0.0% 0.0% 0.0% 



Domain: SEA Monitoring 

Survey Question: Part 6.4) 

Inquiry: Statement on the usefulness of 
ADE program monitoring to help 
districts with their operation of 
Chapter 2 T&PD activities. 





POSITIVE RANGE 


Agree 


Strongly Agree Combined 


53.0% 


42.0% 95.0% 




NEGATIVE RANGE 


Disagree 


Strongly Dim agree Combined 


5.0% 


0.0% 5.0% 



Domain: Quality of SEA Technical 
Assistance 

(Survey Questions: Part 6.2 and 
6.3) 

Inquiry: Statements on the usefulness/ 
appropriateness of technical 
assistance and competence of 
staff working in the ADE 
Chapter 2 Office to answer 
questions regarding LEA 
application for federal funds. 



POSITIVE RANGE 

Agree Strongly Agree Combined 

43.0% 57.0% 1000% 

NEGATIVE RANGE 

Disagree Strongly Disagree Combined 

0.0% 0.0% 0.0% 



Analysis of Area A and Area B 
Discussions. From the information 
collected in this portion c r the Chapter 2 
Coordinator Questionnaire, it can be seen 
that respondents strongly agreed that 
Chapter 2-supported T&PD activities 
provided them with opportunities for 
professional development that otherwise 
would not have been available. 

In addition, 97 percent of the respondents 
agreed that these T&PD programs 
positively impacted at-risk students. 

Interestingly, this impression of the positive 
impact on at-risk students contrasts 
somewhat with the reports from staff who 
indicated that while the T&PD activities 
they attended were worthwhile, parti- 
cipation in the activities did not 
automatically translate into positive 
impacts in the classroom. 



♦ Comments 



The information derived from the Chapter 
2 Coordinator Questionnaire provides 
insight on the administration and operation 
of T&PD programs at the LEA leveL 
Similar to the responses received from the 
individual staff participants, a major focus 
of district T&PD programs is instructional 
delivery. 

In addition, the types of activities made 
available to classroom teachers, school 
administrators and support services 
personnel is not strictly mandated at the 
district level. Rather, 46 percent (19 
districts) place their emphasis on allowing 
participation in in-service activities which 
are initiated and requestedby the individual 
staff person based on his/her unique needs. 

This finding coincides with that of the 
individual course survey results which 
showed a wide variety of descriptive course 
titles and content areas. Thus, Chapter 2- 
funded T&PD activities appear to promote 
and support staff development activities 
which are most needed by the classroom 
teachers, school administrators andsupport 
services personnel. 

Information from this part of the evaluation 
also indicates that districts are utilizing 
procedures to monitor and evaluate T&PD 
activities. This is accomplished through 
use of structured application and evaluation 
methods, district office or committee 
oversight in the planning process, 
committee determination of course offerings 
or by imposing requirements that 
participants must report on, or evaluate, 
the sessions they attend. 

The major points derived from the Chapter 
2 Coordinator Questionnaire are as follows: 

a. All district T&PD coordinators in the 
study have been classroom teachers at 
some point in their careers. However, 
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93 percent indicated that coordination 
and administration of Chapter 2 
programs is not their primary job 
assignment. 

b. Eighty-three percent of the districts in 
the study used committee review teams 
or committee structures in the 
administration of Chapter 2-funded 
T&PD activities including determina- 
tion of general policies and proce- 
dures, formulation of strategic plans 
and the determination of T&PD 
courses offered in the district. 

c. Ninety-four percent of the districts 
which used committees to administer 
Chapter 2-funded T&PD activities 
included teachers on their committees. 
Seventy-seven percent included district 
superintendents, 79 percent included 
principals and 68 percent included 
parents. 

cL The primary focus of Chapter 2- 
funded district T&PD activities was 
reported to be Improved Instructional 
Delivery, at 42 percent of the districts, 
or Increased Staff Effectiveness, at 46 
percent. 

e. Seventy-one percent of the districts 
required certain staff to participate in 
T&PD activities. Most of these 
individuals were newly hired teachers 
or administrators. 

f. Only five of 41 districts (12%) in the 
study reported not utilizing at least one 
of the following procedures for 
monitoring participation and effec- 
tiveness of their T&PD activities: (1) 
requiring a formal application process 
for staff to participate in T&PD 
activities; (2) undertaking a formal 
review process of applications 
submitted; (3) undertaking formal 
evaluations of district T&PD courses 
staff attended; or (4) requiring all 
participants in district T&PD activities 



to submit evaluations of courses 
attended. 



Three of these five districts contained 
2,000 or fewer students while only one 
contained more than 15,000 students. 
However, all of the districts which 
reported no formal systems for 
monitoring participation utilized com- 
mittee review teams Ao develop the 
policies and strategic plans for their 
Chapter 2-funded T&PD program, 
reviewed applications of staff wishing 
to attend in-services and determined 
the type of T&PD courses offered. In 
addition, all but two districts in the 
study reported utilizing some type of 
evaluation of T&PD activities to obtain 
feedback on training sessions attended 
by district personnel. 

The data compiled from the Chapter 2 
Coordinator Questionnaire reveal an 
overall positive view of LEA involvement 
in Chapter 2-supported T&PD activities. 
The responses disclose an emphasis on 
training in instructional delivery and 
classioom skills; identification of a mix of 
district- and staff-defined professional 
needs; widespread use of a variety of 
evaluation methods, both formal and 
informal; and a commitment to providing 
in-service training to new teachers and 
administrative staff. In addition, over- 
sight committees for T&PD programs at 
the local level included teacher, staff and 
parent representation in a majority of 
districts. A 



C. Changes in Local Educational Services 



♦ Magnitude and Key 
Patterns of Change 



Further review on page 4 of Table 1, 
Three Year Summary of Targeted 
Assistance Areas, offers the reader a 
historical perspective on the spending 
decisions and participation of local 
educational agencies in Arizona's Chapter 
2 Program. 

The number of districts operating Chapter 
2-assisted programs of Training & 
Professional Development grew, as 
indicated below, from a total of 63 in SY 90 
to 79 in SY^ 



Chart5 
Districts Participating in 
Chapter 2-Funded T&PD Programs 
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In SY 91, the period during which the 
state's evaluation was conducted, T&PD 
activities in 66 participating districts 
extended direct or indirect influence over 
immense populations of students and 
educational personnel throughout the 
state. Looking again on page 10 at Table 2, 
Characteristics of Districts with Chapter 2- 
Funded T&PD Programs, reveals the size 
of populations belonging to either study- 
group or non study-group districts. 

Th3se 66 entities comprise 78 percent of 
Arizona's student enrollments, 64 percent 
of all public schools, 76 percent of certified 
staff and an equal share of all current 
public school teachers. The magnitude of 
this potential for impact is unquestioned. 
Given these proportions, it is clear that 
Chapter 2-funded T&PD activities have 
the potential to influence Arizona's 
teachers, students and school-based 
programs. 



Sounc ChapUr 2 Annual Bfmaml Pn^cU AppUa'JMK BY 90, 91 ud 92. 

Growth of this type (i.e., an increase in 
participation of more than 25% in two 
years) demonstrates the emphasis that 
autonomous districts collectively ascribe 
to T&PD activities. 
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Chart 6 
LEA Allocations for 
Chapter 2-Funded T&PD Programs 
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Arizona 
districts 
which operate 
Chapter 2- 
funded T&PD 
programs 
comprise 78 
percent of the 
state's student 
enrollments. 
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In the first 
three years of 
ESEA, 
researchers 
noted a 
marked 
increase in 
focus on 
T&PD 
programs by 
an ever- 
increasing 
number of 
districts 
across the 
state. 



ADE researchers examined T&PD activities 
with an eye to trend data on file in the ADE 
Chapter 2 Office. In terms of local 
allocations, T&PD programs received 
$1,915,976 in SY 90. By SY 92, the amount 
grew to $2,127,466. This represents a 
growth of 11 percent in just two years, 4 
percent above the rate of inflation* during 
this time. 

Because the distribution of budgeted 
funds is driven by cumulative district 
demand, the figures in Table 1 demon- 
strate emphatically the increasing priority 
placed on staff development by Arizona 
educators. Indeed, in SY 90 T&PD pro- 
grams accounted for approximately 36 



the part of classroom teachers or is 
contingent upon policy decisions made by 
district administrators, the resulting effect 
is a marked increase in focus on T&PD 
programs by an ever-expanding number of 
districts across the state. 

While Table 1 reveals the growing priority 
being given to T&PD programs, it can not 
convey the impact this additional funding 
has had upon teachers and students. The 
direct measure of these effects is evident in 
Arizona's evaluation of local Chapter 2 
T&PD activities funded during SY 91. 

In preparation for this evaluation, ADE 
Chapter 2 Office staff reviewed the types of 



Chart? 

SY 91 Staff Participation in Full-Time Equivalents 



32.5 



33.8 17.9 46.7 




CD Administrators 
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■ Teachers Aides 

ESI Support Services Staff 

EB Clerical Staff 



613.3 



Teachers comprise the majority of educators 
being assisted under Chapter 2 



Sourot: Arisen* Department of Education Chapter 2 Office. Annuel Specie! Project* Application*: School Yean 1960/90, 91 and 92. 



percent of all budgeted Chapter 2 funds 
while in SY 92, this proportion had grown 
to over 38 percent. 

Whether such expanding emphasis is 
viewed as a result of stronger demand on 



• Based on the historical (4.2% in 1991) and forecast 
(3.0% in 1992) figures for the Consumer Price Index as 
reported in "Arizona Business," Arizona State Univer- 
sity, College of Business Research, Tempe, Arizona — 
April 1992. 



T&PD activities districts reported at the 
times they applied for Chapter 2 funding 
between 1988 and 199 1. From this research, 
a list of Course and Training Components 
was developed for survey at the local level. 
This list contained 54 course titles, 
partitioned within 11 content components 
that have come to be associated with 
Chapter 2-supported Training & 
Professional Development activities over 
the years. 



66 



Interestingly, the survey responses revealed 
a wider variety of local course offerings 
than anticipated. In response to questions 
CO ntamedmiheLocalChapter2Evaluation 

Survey, district trainees reported taking a 
total of 83 separate courses. This 
repref ented 29 course titles in addition to 
those which had been previously detailed 
within the annual Special Projects 
applications. 

Because applications may be planned as 
much as a year before delivery of training, 
this survey finding was notable in terms of 
the program flexibility it suggested. 
Furthermore, the finding was consistent 
with the growing number of districts 
participate g in T&PD activities as well as 
the trend of directing a larger share -f 
Chapter 2 funds toward T&PD programs. 

The variety of course offerings observed 
from this evaluation is important to an 
understanding of the overall impact which 
T&PD activities exert upon students and 
teachers. It is evident as well that local 
Chapter 2-funded T&PD activities can 
provide one-of-a-kind training conduits for 
teachers and administrators in their pursuit 
of instructional or managerial knowledge 
and skill development. 

From the data collected, it is clear that the 
emphasis on T&PD programs is increasing 
throughout Arizona. More districts are 
supporting their T&PD activities with 
Chapter 2 funds while longer-term 
participants offer ever-widening access to 
administrators and support services staff. 
At the local level these factors represent 
the key patterns of change. 

At the state level, changes in local education 
services caused by the use of Chapter 2- 
funded T&PD programs are directly 
observed tough the inauguration of the 
Arizona Student Assessment Program 
(ASAP). The availability of Chapter 2 
monies provided the framework for 
enactment of fundamental changes in 
classroom learning. 



ASAP, an innovative, performance-based 
instructional and assessment system, aligns 
content, teaching methods and assessment 
of student performance to essential basic 
and higher-order skills in reading, language 
arts and math. 

In addition to ASAP representing the most 
far-reaching and complex undertaking of 
systemwide assessments in the country, 
Arizona's model fundamentally differs from 
that of other states in that ASAP creates a 
complete alignment between mastery of 
essential skills, student competency 
assessments, the instructional focus of 
teachers and measurable, as well as 
reportable, outcomes. 

This innovative endeavor, under its 
operational umbrella: Goals For 
Educational Excellence, represents 
Arizona's primary response to the national 
Effective Schools mandate. ASAP was 
designed to meet several goals: 

• fundamental shifts away from nationally 
normed testing toward performance- 
based assessments 

• reform of instructional dehvery methods, 
and 



Researchers 
found that 
local Chapter 
2-funded 
T&PD activi- 
ties can 
provide one- 
of-a-kind 
training 
conduits for 
educators in 
their pursuit 
of skills 
development. 



• alignment of curriculum against requisite 
basic and higher order skills, mastery of 
which the State Board of Education 
considers essential in grades K-12. 

To ensure success in fully implementing 
ASAP and thereby realizing a majority of 
its Chapter 2 goals, the Arizona Department 

of Education launched a multi-year, 
transitional program of Training & 
Professional Development. Representatives 

of all public school districts in Arizona as 
well as ADE staff who worked as technical 
advisors in initiating ASAP, were afforded 
these Chapter 2-supported training 
activities. 

State-administered Chapter 2-funded 
T&PD monies supported the staff devel- 



LEA staff 
reported a 
significant 
improvement 
between 
October 1990 
and March 
1991 in the 
effectiveness 
of ASAP- 
related T&PD 
provided by 
theADE. 



opment and district in-service training 
required to construct and implement 
Arizona's ASAP program. In March 1992, 
150,000 students in grades 3, 8 and 12 
participated in the pilot stage of ASAP 
implementation. 

Involvement in the assessment portion of 
ASAP for the first time included bilingual, 
ESL, Special Education, and other 
handicapped students who were, in many 
cases, previously exempted from traditional 
norm-referenced testing. 

Beginning in September 1992, all teachers 
and students in the state will experience 
the impact of some aspect of ASAP as the 
program attains full integration within the 
state's educational system. 

Successful implementation of the Arizona 
Student Assessment Program testifies to 
th* effectiveness of the Chapter 2-funded 
T&PD activities provided in SY 91 to state 
ASAP liaisons and local district repre- 
sentatives. Survey responses from those 
participating in the fall 1990 ASAP 
conference and/or the spring 1991 
workshops measure many observable, 
positive impacts. District staff reported a 
significant improvement between October 
1990 and March 199 1 in the effectiveness of 
ASAP-related T&PD provided by the 
Arizona Department of Education. 

Approximately 30 percent of LEA 
participants gave ADE facilitators a 
negative rating overall on fall conference 
effectiveness. However, this measure of 
dissatisfaction diminished to an 11 percent 
negative rating for the spring workshops. It 
was observed from ASAP Liaison 
Questionnaire responses that, during this 
interval, ADE staff were undergoing 
substantial Chapter 2-supported ASAP 
training of their own. 

It is also clear from the Survey of Conference 
Effectiveness results, as well as direct 
observations of the evolution of ASAP in 
Arizona, that Chapter 2-funded T&PD 



activities provided at the state level were 
instrumental to the successful imple- 
mentation of the program. 

ASAP-related Training & Professional 
Development became the dominant vehicle 
for arriving at educational innovation and 
systemic reforms. As a result, ASAP now 
exerts a positive influence on virtually every 
student and teacher in Arizona* Put simply, 
it provides the foundation for a new and 
more effective learning environments in 
our state's public schools. $, 
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D. Effects upon Arizona's Students and Teachers 



♦ Magnitude and Key 
Patterns of change 



0 

ERIC 



The SY 91 Arizona examination of 
Chapter 2-funded T&PD activities 
provided direct observations on the scope 
and potential impact of these activities on 
students and teachers in Arizona. 

Without question, the demographics of 
districts receiving T&PD funds affirm that 
a substantial proportion of students, 
teachers and administrators in Arizona are 
affected by these locally sponsored T&PD 
programs. 

Overall, 66 districts in Arizona received 
Chapter 2 funding for T&PD programs in 
SY 9 1. These districts comprised 78 percent 
(530,597) of total student population in the 
state, 76 percent of all teachers (27,479) 
and 71 per sent of all administrators (1,595). 

ADE's Local Chapter 2 Evaluation Survey 
collected direct responses from LEA staff 
within a study group composed of 43 of 
these 66 districts. Prior to administering 
the survey, researchers asked all 43 district 
T&PD coordinators to provide complete lists 
of their staff who had attended Chapter 2- 
funded training during SY 91. 

These individuals were further categorized 
by job title as administrators, instructional 
personnel or support services staff. From 
information provided before the survey was 
given, researchers determined that a total 
of 5,139 classroom teachers, or 21 percent 
of all teachers in the study-group districts, 
had received Chapter 2-supported training 
and that these instructional personnel 
impact an estimated 128,500 to 270,000 
students, implying a direct coverage of 



between 19 and 40 percent of total student 
enrollment in Arizona. 

Unfortunately,itis not possible to determine 
how many teachers were trained or how 
many of their students were affected in the 
remaining 23 districts; however, T&PD 
programs in these districts were believed 
not to be as extensive as those in the 43- 
member study group. 

Potential Impact on At-Risk Students. 
As previously noted in Part II.B, a 
substantial number of students "at i*isk n of 
failure or dropping out of school were 
instructed by teachers who had attended 
Chapter 2-funded T&PD activities. 

Based upon responses returned by 804 
teachers sampled from the study group, 
approximately 3,312 at-risk students are 
impacted by Chapter 2-funded T&PD 
programs. 

Using statistics yielded by the teacher 
sample, this implies an overall coverage in 
the 43-district study group of between 
32,900 to 44,500 at-risk students who are 
receiving the benefits of their 5,139 teachers' 
attendance at Chapter 2-funded T&PD 
activities. Unfortunately, no reliable 
estimates are available on the total number 
of at-risk students contained either in the 
66 Arizona districts receiving Chapter 2 
T&PD funding or in the state as a whole. 

In addition, 97 percent of the respondents 
attributed positive impacts for at-risk 
students to these T&PD programs. 
Interestingly, these impressions of positive 
impact on at-risk students contrast 
somewhat with other survey results from 
the course evaluation questions. 

The data indicated that while most staff 
felt T&PD activities they attended were 
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worthwhile, over 20 percent reported that 
participation in the activities did not 
automatically translate into positive 
impacts in their classrooms. 

The figures presented in Table 3 reveal 
that the 43 Chapter 2-funded T&PD 
districts selected for the evaluation study 
represent the majority of students, schools 
and staff within the state. 

This underscores the enormous potential 
Chapter 2-funded T&PD programs have 
for positively affecting teachers, school 
administrators and, most importantly, the 
children of Arizona. 

With T&PD activities occurring in such a 
large percentage of Arizona districts, the 
direct or measurable outcomes ofin-services 
may greatly understate the longer-term 
impacts on students and staff. 

Job-related contacts of staff who have 
attended Chapter 2-supported in-services 
with other professional staff ensures that 
participants will share their enhanced 
knowledge and new skills with their 
colleagues — to a greater or lesser degree. 
This sharing broadens and deepens the 
impact of the original training and creates 
a ripple effect which extends throughout 
the state, continuing for an undetermined 
period of time. 

From the information provided by survey 
respondents, it is evident that the overriding 
demand for Chapter 2-funded T&PD 
programs in Arizona revolves around 
training in instructional delivery. Seventy- 
two percent of Local Chapter 2 Evaluation 
Survey respondents indicated their primary 
reason for attending T&PD activities was 
to acquire more effective instructional 
delivery skills. 

In addition, Essential Elements of 
Instruction was revealed as the course most 
often attended by respondents, repre- 
senting 12 percent of the 1,711 course 
evaluations returned. 



Given evidence of the large proportions of 
regular and at-risk students served by 
teachers who have attended T&PD 
programs and discovery of the heavy 
emphasis placed on acquisition of 
instructional skills, it is apparent that 
Chapter 2-funded T&PD programs in 
Arizona have positively impacted amajority 
of students through basic improvements in 
instructional methodologies. 4 
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E. T&PD Effectiveness Indicators 



1. Chapter 2-Supported T&PD Activities under the 
Arizona Student Assessment Program (ASAP) 
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Omitted from the discussion in 
Section D regarding the effects upon 
Arizona's students and teachers, was the 
magnitude of direct and indirect effects 
which had been observed as a result of 
providing Chapter 2-funded T&PD 
activities to ADE staff in charge of 
implementing the new student assess- 
ment program. 

The direct impacts were measured through 
a survey of LEA participants in state- 
sponsored ASAP conferences and 
workshops. The indirect impacts were 
derived from pertinent observations of the 
ASAP implementation in schools and 
classrooms throughout the state. 

As documented in Part HA, SEA Chapter 
2 monies supported staff development 
activities for state ASAP liaisons andfunded 
a conference and a series of workshops on 
the new assessment program. ADE staff 
effectiveness was measured directly by LEA 
participants attending these state- 
sponsored workshops. 

As demonstrated in the adjoining data set, 
respondents to the Survey of Conference 
Effectiveness gave increasingly positive 
ratings to ADE staff facilitators and to 
the overall effectiveness of the sessions as 
training progressed from the introduction 
of ASAP in October 1990 through June 
1991. 

At the time of the fall 1990 ASAP conference, 
ADE staff were undergoing their own 
Chapter 2-supported training and the ASAP 
program was still in the developmental 
stage. This explains the lower scores given 



to the facilitators and the effectiveness of 
the sessions. However, by the spring of 
1991, with the benefit of Chapter 2-funded 
T&PD assistance, ADE staff were able to 
provide higher levels of support and to 
supply more complete information to LEA 
representatives participating in regional 
workshops. 



1990 ASAP Conference 


Category 


Negative 


Positive 


Response 


Response 


Facilitators 


30% 


70% 


Overall 






Effectiveness 


20% 


80% 


1991 ASAP Workshops 


Category 


Negative 


Positive 




Response 


Response 


Facilitator* 


11% 


89% 


Overall 






Effectiveness 


11% 


89% 

■ 
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As mentioned elsewhere in this report, the 
indirect impact of the T&PD activities 
evaluated at the state level were seen in the 
successful implementation of a radically 
new performance-based instruction and 
testing program in Arizona. 

In March 1992, approximately 150,000 
students in grades 3, 8 and 12 participated 
in an ASAP pilot test and acquired their 
first familiarity with performance-based 
assessment testing. By September 1992 
every teacher, every grade of students and 
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every Arizona school will be impacted by 
this new program which more closely aligns 
curriculum, reforms instructional strategies 
and assesses student performance. 

A direct measure of student progress cannot 
be quantified within the limited scope and 
12 month time frame of this evaluation, but 
the positive effects of aligning the Arizona 
educational system with a skill-based 
curriculum, effective instructional delivery 
and continuous student assessments of 



essential skills are apparent and cannot be 
understated 

Funding made available by the SEA 
through Chapter 2 for the tremendous 
amount of staff development that was 
necessary for ASAP at both local and state 
levels was, and remains, critical to the 
success of this vital new program. Without 
Chapter 2 funding, it is doubtful that ASAP 
could have progressed to the extent that it 
has. 



2. Chapter 2-Supported T&PD Activities within Local 
Educational Agencies 



Responses by district T&PD coordinators 
to the Chapter 2 Coordinator Questionnaire 
indicated another direct impact of Chapter 
2-funded T&PD activities. In this case, the 
impact is relevant to overall school quality 
both from an instructional point of view 
and from the perspective of school 
environment. 

Eighty-three percent (34 of 41 districts) 
surveyed indicated that they employed a 
committee structure to oversee various 
aspects of their T&PD programs. The 
membership of these review committees 
varied However, teachers were represented 
on committees 94 percent of the time ; district 
superintendents, 77 percent; school 
principals, 79 percent and parents, 68 
percent. 

This combined involvement of teachers, 
administrators and community members 
at the planning and administrative stag^ 
ofT&PD activity selection suggests a strong 
commitment to schoolwide improvement. 

In this shared environment, teachers and 
local administrators are interacting with 
district policymakers to implement 
programs that impact students in the 
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classroom. As a result, teachers and their 
immediate supervisors become collectively 
as accountable lor tne success or failure of 
the training programs as were the central 
decision-makers. 

Finally, the notable involvement of parents 
on such a large proportion of the Chapter 2 
review committees suggests a strong 
commitment on the part of study-group 
district to the securing of community 
support for the policymaking processes of 
schools and the district. 

Such commitment acts to empower parents 
to provide input on how their children's 
schools are run. Similarly, parental 
involvement also indicates parental 
accountability in the success or failure of 
the committee review structure in 
strengthening local T&PD programs. 

In addition to the large number of teachers 
and administrators that voluntarily 
participated in T&PD activities, 71 percent 
of the district coordinators reported that 
newly hired teachers and administrative 
staff are required to attend specific Chapter 
2-fimded T&PD courses. 
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The majority of these courses focused on 
acquisition of instructional skills such as 
Essential Elements of Instruction and 
thinking skills; in science, language arts 
and adaptive curriculum; and knowledge 
about Effective Schools. 

Considering the significant number of 
students directly affected by teachers 
attending Chapter 2-funded T&PD 
activities, the number of classroom teachers 
taking advantage of such staff development 
opportunities, the emphasis placed by 
teachers and district planners on regularly 
updating their instructional skills and the 
tremendous impact of the state's ASAP 
initiative, it is readily apparent that the 



Chapter 2 program and its support of 
selected targeted assistance areas is 
essential to educational improvement in 
Arizona- 
Chapter 2 funding clearly impacts students 
and teachers in the classroom. While the 
statistical results noted earlier in this 
report cannot verify directly observable 
improvements in students* test-scores, life 
skills or their likelihood of future success 
(this is beyond the scope of a one-year 
study), it is abundantly clear that such 
impacts do occur and it is reasonable to 
attribute these positive benefits to effective 
state and locally administered T&PD 
programs supported with Chapter 2 funds. 



3. Ancillary Benefits Derived from the Evaluation 
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Undertaking the Evaluation of Chapter 2 
Program Effectiveness produced 
unexpected benefits that were incidental to 
the study itself but of great value to the 
ADE. These findings are presented below. 



A Listing of Ancillary Benefits 

T Arizona's Evaluation of Chapter 2 
Program Effectiveness marked the first 
analysis ADE conducted of such a highly 
representative sample of the state's 
public schools. In addition, the number 
of survey responses were sufficiently 
high to provide districts with valid data 
and analyses of their individual 
programs of Training & Professional 
Development, 

V The Local Chapter 2 Evaluation Survey 
established the first reliable estimate of 
average class size in Arizona's public 
schools. 

▼ This permitted assertions regarding 
the number of students impacted by 
teachers who participated in local 



Chapter 
programs. 



2-supported T&PD 



▼ This also permitted assertions 
regarding the number of at-risk 
students impacted by these teach- 
ers, which is relevant in terms of 
National Education Goal No. 2, 
stating that by the year 2000, the 
high school graduation rate will 
increase to at least 90 percent. 

V The Local Chapter 2 Evaluation Survey 
was instrumental in the defining the 
variety, content, and focus of Chapter 2 
T&PD courses taken throughout 
Arizona. 

▼ TheLocalChapter2EvaluationSurvey 
established the extent to which T&PD 
programs emphasized instructional 
delivery and confirmed that this focus 
is not policy-driven, but demand-driven. 

▼ The Chapter 2 Coordinator Question- 
naire discovered the depth and 
breadth of interaction between the 
stateChapter20£5ceanddistrictT&PD 
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coordinators, and among district T&PD 
coordinators, trainees and review 
committees. 

▼ This survey also generated quanti- 
tative data on the extent of syste- 
matic consultation, a federally 
mandated responsibility, which takes 
place. 

▼ The survey affirmed a strong 
correlation between planning, design 
and districtwide consultation 
activities occurring in the operation 
of T&PD programs as opposed to 
that which was delineated for the 
other targeted assistance areas. 

t The survey also revealed the 
enormous potential for community 
involvement in schoolwide improve- 
ment possible through existing 
review committee membership (i.e., 
parents, principals, teachers, and 
district administrators). 

T The Chapter 2 Coordinator Question- 
naire disclosed that 29 of 41 districts 
responding to the survey mandated 
specific in-services for newly hired 
educators. Twenty-six of these 29 
districts scheduled T&PD activities for 
new teachers while 17 of them required 
newly hired principals and assistant 
principals to attend employee 
orientation and in-service training. 

▼ Analysis of the demographics associ- 
ated with the 43-district study group 
showed that, as a percentage of state 
totals, these districts encompassed 
over 70 percent ofboth Arizona students 
and certified district staff, thereby 
ensuring that the wealth of data 
collected will be of interest to research- 
ers for years to come. 

T 1 e evaluation produced the first 
statewide view of district T&PD 
management and accountability 



procedures, such as in-house evalua- 
tions of training applications or 
attendance patterns and utilization of 
this information for policy formulation 
and strategic planning. 

▼ The Arizona Student Assessment 
Program was clarified for the first time 
in terms of National Education Goal 
No. 3, stating that by the year 2000, 
students will leave grades 4, 8 and 12 
having demonstrated competency in 
English, mathematics and science. 

▼ The evaluation pointed to observable 
impacts of staff development activities 
affecting progress in implementing 
ASAP. This was notable in that it 
represented the first formal feedback 
received for SY 90 and SY 91. 

As a final note on peripheral benefits 
accrued as a by-product of the Evaluation 
of Chapter 2 Program Effectiveness , ADE 
discovered that significant progress could 
be achieved toward future refinement of 
ASAP by analyzing the ASAP Liaison 
Questionnaire data in concert with the data 
obtained from the Survey of Conference 
Effectiveness to avoid documented pitfalls. 

Specifically, the evaluation results affirmed 
that increasing the level of involvement 
and training of ASAP staff in initial goal- 
setting and problem-solving processes 
would minimize problems associated with 
translating program goals and objectives 
into specific implementation tasks and 
maximize the potential for successful 
outcomes. 

Gaining this perspective is crucially 
important to state administrators due to 
the complexity of ASAP and its pervasive 
influence on Arizona pupils. As the ASAP 
approaches ifcs first full year of operation in 
SY 93, the staff development process at 
both the LEA and SEA levels becomes of 
paramount importance to achieving salient 
educational goals. A 
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Report Summary 



Tie significant effort and investment 
that the Arizona Department of 
Education put into the Evaluation of 
Chapter 2 Effectiveness resulted in the 
accumulation of a tremendous amount of 
information and knowledge. Compilation 
of this data has, for the first time, enabled 
education professionals, at both state and 
local levels, to document the quantitative 
and qualitative aspects of their Chapter 2 
T&PD programs. 

Prior to this evaluation, the scope and the 
impact of Chapter 2-funded T&PD 
activities upon teachers and admin- 
istrators; at-risk and mainstream students; 
school and classroom environments; and 
school improvement efforts were not 
known. Likewise the importance of T&PD 
efforts to the successful implementation of 
ASAP was not known. 

From the information gathered for this 
report, we have gained a fundamental 
understanding of the benefits derived from 
Chapter 2-funded T&PD programs. The 
benefits documented by our evaluation 
include, but are not limited to, the following 
six items: 
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Through their requests for, and 
participation in, certain Chapter 2- 
funded T&PD activities classroom 
teachers demonstrated their need 
for updated training in instructional 
delivery skills and content specific 
knowledge* It is apparent that these 
T&PD programs serve as a vital 
resource in ensuring that teachers 
are provided with the necessary 
training and knowledge they need to 
be effective in the classroom. 

The instructional skills and content- 
related knowledge delivered to 
classroom teachers through 
Chapter 2-funded T&PD programs 



directly impact upwards of 270,000 
students in our K-12 public school 
system, including 44,500 pupils 
considered to be at risk of not 
completing high school. Without 
Chapter 2 funding, many educators 
would be deprived of crucial 
opportunities for e nhan cing their 
skills and knowledge. 

Given the broad scope of student 
coverage engendered by these T&PD 
programs, it is evident that reduction 
or removal of such important staff- 
development opportunities would 
negatively affect a major segment of 
Arizona's public school students. 

Although just 66 of 220 Arizona 
school districts allocated Chapter 2 
funds for T&PD activities, these 
districts comprised approximately 
75 percent of all teachers and 
students in the state* Interactions 
among staff, whether formal or 
informal, perpetuated and amplified 
initial training experiences through- 
out each participating district. 

This evaluation clearly demonstrates 
the surprisingly extensive coverage 
of these Chapter 2-supported T&PD 
programs vie we d from the 
perspective ofboth direct andindirect 
participation. 

Information gleaned from the four 
evaluation surveys pinpointed the 
extent to which Chapter 2-funded 
T&PD programs provide oppor- 
tunities for educators to participate 
in a wide variety of staff development 
activities, However, this general 
observation understates the real 
importance of cur findings — the 
revelation that the structure of these 
T&PD programs is highly significant. 
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It is the structure which allows 
teachers and administrators to 
target their comprehensive staff 
development and training needs to 
the specific areas in which they are 
the most inadequate. If funding 
flexibility were restricted or limited 
to specific training areas, the 
program would be unable to fulfill 
effectively the needs of Arizona's 
educators. 



Our evaluation found that Chapter 
2-funded T&PD programs 
contributed positively to overall 
school improvement efforts and to 
the participating facility's general 
learning environment. 

It also disclosed that these programs 
encourage and promote involvement 
of parents and other individuals from 
outside-the educational co mm u ni ty. 

Study data revealed that inclusion 
of teachers, school and district 
administrators, and parents on 
committee review teams overseeing 
the planning and operation of 
Chapter 2-funded T&PD programs 
greatly added to the cooperative 
environment of the school or district. 

This school improvement aspect 
positively influenced the attitudes 
and expectations of classroom 
teachers and school administrators. 

Participants in the Arizona 
evaluation concluded that this 
cooperative involvement lead to 
improved classroom environments. 

They agreed that the potential for 
better student outcomes would 
increase as a result of a more 
knowledgeable staff; enhanced 
teaching skills; and increased 
interaction between parents, 
teachers and school administrators. 



Staff development activities partially 
supported by Chapter 2 monies 
helped inaugurate the Arizona 
Student Assessment Program, one 
of the most significant reforms ever 
undertaken in Arizona's public 
education system. This evaluation 
determined that the SY 91 Chapter 
2 T&PD program dollars spent in 
support of pilot testing Arizona's new 
student assessment initiative 
significantly contributed to success 
of its later statewide implementation. 

The availability of Chapter 2 monies 
for training SEA and LEA staff 
enabled the implementation to 
proceed in an effective and efficient 
manner. Although this SY 91 
evaluation denotes specific findings 
tied to the ASAP initiative, more 
importantly, it demonstrates the 
tremendous utility of Arizona's 
Chapter 2 T&PD program in 
facilitating critical and positive 
changes in the state's public 
education system. 



Without question, this report contains a 
wealth of information, both quantitative 
and qualitative, on the scope of training, 
student coverage and demonstrated impact 
of Chapter 2-funded T&PD programs on 
Arizona's public K-12 school system. 

The results of this evaluation are both 
profound and conclusive: profound from 
the standpoint of understanding the extent 
to which these T&PD activities influence 
nearly every level of Arizona's educational 
system (students, teachers, and school and 
district administrators). And, they are 
conclusive in terms of the magnitude of 
local and state staff participation, the 
coverage of mainstream and at-risk 
students, the scope of skill development 
and content areas supported, and the 
indirect benefits accorded to non- 
participants within Chapter 2-supported 
local educational agencies. 4 
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The Governor's Chapter 2 Advisory 
Committee comprises 14 members, 
residing in urban and rural Arizona. 
Committee membership meets require- 
ments for collective . representation of 
educational interests outlined in ESEA- 
Chapter 2, Section 1522(aX2). 

Arizona's advisory committee establishes 
formulas for allocation of funds to local 
educational agencies; advises the Arizona 
Department of Education (ADE) on state 
use of funds among targeted areas; author- 
izes and oversees competitive awards for 
school-based. Effective Schools projects; 
and otherwise participates in the 
planning, initiation and evaluation of 
state-administered Chapter 2 programs. 

The report in which this appendix appears 
embodies active consultation by advisory 
committee members in terms of their 
review and comments as prescribed by 
Section 1522(aX6XB). As early as its May 
1990 meeting, members were beginning to 
discuss peripheral responsibilities 
mandated by Congress in terms of their 
evaluating the program's benefits and 
other notable impacts upon Arizona's 
educational programs. 

This evaluation, while significant in 
viewing the Chapter 2 Program in its 
entirety, is limited to the committee- 
approved selection of one of six areas of 
targeted assistance: Programs of Training 
& Professional Development (T&PD). 

Presentations by the Chapter 2 Office 
between May 1990 and October 1991 
focused upon the extent of programmatic 
changes occurring over the first three 
years of current legislation. Committee 
members pondered allocation trends, model 
program successes, annual statistical 
reports and updates on the progress of the 
evaluation itself. In October, the Advisor- 
Committee heard the results of Arizona's 
four state and local surveys. Mr. Edward 
F. Sloat, Deputy Associate Superintendent 



for Research and Development, presented 
preliminary findings that would later be 
published in the Evaluation of Chapter 2 
Program Effectiveness. 

The final draft of this document was 
ratified at the committee's regularly 
scheduled meeting in January 1992 while a 
select subcommittee continued to monitor 
content throughout the publication pro- 
cess, thereby completing its review prior 
to printing and submission of the report. 

During the October and January sessions, 
lively discussions centered upon individual 
evaluation outcomes. Members were 
intrigued by complexities uncovered in 
the effects of state-administered training 
upon ADE staff. As a prelude to obtaining 
their comments, advisory committee 
members heard the outcomes of collective 
responses by district staff who attended 
training workshops on the Arizona Student 
Assessment Program (ASAP). 

What members learned was contrasted 
against results of interviews administered, 
in a subsequent survey, to ADE staff who 
assisted in implementing ASAP through- 
out Arizona's public school districts. Of 
primary interest was the dramatic manner 
in which perceptions changed over time as 
16 ADE specialists, 11 of whom were 
interviewed, gained sufficient knowledge 
to effectively assist districts in an expedient 
inauguration of ASAP. 

Although deliberations were, at times, 
equally divided between survey results at 
the state and local levels, committee 
members responded on record by 
interjecting personal insights into findings 
attributable to both of the local surveys. 

At the local level, questionnaires were sent 
to district personnel representative of 
Arizona educators who had attended staff 
training sessions during SY 91. This 
survey was augmented by a second survey 
administered to a group of district T&PD 
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coordinators. Responses from T&PD 
coordinators further qualified the "depth of 
district programming." Members began to 
appreciate the magnitude of impacts being 
made by locally administered T&PD 
activities in SY 91 upon students w>o were 
taught, counseled or otherwise directed by 
the nearly 31,000 LEA trainees statewide. 

With interest increasing for the surveys 
conducted at the local level, the advisory 
committee found common ground in two 
assertions Iveing considered as part of the 
final draft. First, the advisory committee 
reacted immediately to characterizations 
of course-by-course quality indicators 
found in questions 8, 9 and 10 on page 4 of 
the Local Chapter 2 Evaluation Survey 
(see Appendix 5). Representing a string of 
composite responses, members heard that 
a statistically significant progression had 
been discovered. 

Initial analysis suggested that largely 
positive evaluations of instructors, course 
content and trainee handouts had 
deteriorated into a less positive impression 
regarding classroom teachers' "ability to 
fully implement" the training they received 

Four members with many accumulated 
years of teaching experience offered strong 
rebuttal to that which was being 
interpreted as a a negative departure" in 
the course-by-course quality indicators. 
All suggested that "classroom teachers 
need time" to process new instructional 
delivery techniques. Where "time for 
reflection" was absent from in-service 
agendas, perceptions regarding 
applicability of training, regardless of 
peripheral benefit, "tended to drop." 

Committee members also felt a certain 
disenfranchisement frequently occurs 
"where decision-making and scheduling of 
T&PD activities reside exclusively at the 
administrative level." Members agreed 
with researchers that the inability to 
study impact over a longer time period was 



an inherent weakness of this exercise. One 
member discussed "on-time" surveys, 
administered as district trainees departed 
training. Although mentioned as a 
possible alternative to the evaluation 
method actually used, it was quickly 
discounted as "impossible to coordinate." 

Finally, members contradicted conclusions 
being put forth regarding the responses 
given by districts to question 2 of the 
Chapter 2 Coordinator Questionnaire 
(see Appendix 6). 

Twenty-six of the 41 T&PD coordinators, 
who responded to this inquiry indicated 
that their districts do not require a 
"formal application" for training. Data in 
the draft report suggested that only 37 
percent of coordinators had access to 
"procedures and quality reviews which 
systematically collect, maintain and 
evaluate participation in training 
activities." As a result, some advisory 
committee members questioned the 
accuracy with which the term, formal, 
was being locally interpreted. As members 
pointed out, more than 50 percent of the 
same study-group districts maintained 
oversight committees which reviewed 
requests for training and determined course 
offerings. 

These two distinctions seemed to cast 
doubt upon the significance of the dis- 
cussion as it appeared in the draft report. 
Members were reminded that districts are 
not obligated to initiate a "formal process 
for approval" of their Chapter 2-assisted 
training courses. There was general 
agreement that, although not a requirement 
imposed by the state, a low YES response 
(37 percent) to this question suggested dis- 
tinct possibilities: districts maintain 
sufficient safeguards, yet T&PD coordi- 
nators judge systems to be unstructured] 
some districts do not maintain trainee 
selection systems which can be easily 
evaluated; or additional monitoring by SEA 
staff may be warranted as a result of this 
inquiry. 
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Wiile the processing for most of the 
information collected as a result of 
the Evaluation of Chapter 2 Program 
Effectiveness was done by Arizona 
Department of Education (ADE) stafffrom 
the Research and Development Division 
(R&D), preparation and development of 
data collection instruments and the 
analysis of the results was a collective 
effort between the Department's R&D and 
Chapter 2 personnel* In addition, input 
from district Chapter 2 coordinators 
throughout the state helped to improve the 
content validity and data integrity of the 
evaluation project. 

As described below, all of the survey 
instruments went through a series of 
development and processing activities to 
ensure the highest possible data integrity . 
This process began with initial discussions 
on which areas of T&PD activities to 
evaluate and continued through to the 
analysis and production of the final report. 

The evaluation utilized four separate 
survey instruments to collect information 
on Chapter 2-funded T&PD activities: the 
Local Chapter 2 Evaluation Survey, the 
Chapter 2 Coordinator Questionnaire, the 
Survey of Conference Effectiveness and the 
ASAP Liaison Questionnaire. The last 
instrument was used to conduct interview 
sessions with ADE staff. 

Methods Used 
in the Development of 
Evaluation Instruments 

First, interviews with individus Is from the 
target survey groups were conducted This 
activity provided ADE R&D and Chapter 2 
staff with an opportunity to clarify the 
issues and understand the activities of the 
target group more clearly. 

For the Chapter 2 Coordinator Question- 
naire, R&D staff conducted preliminary 
interviews, either in person or by phone, 
with nm* Chapter 2 district T&PD 



coordinators. The discussions focused on 
activities, policies and procedures of 
administering local Chapter 2-supported 
T&PD programs. From these interviews, a 
profile of pn*.sss, procedural and policy 
interactions at the district level was 
developed. 

In Part II.B, page 47, the reader is 
provide d with a diagram which presents 
the underlying interactions and involve- 
ment of district administrative staff, the 
membership and the activities of oversight 
c ommit tees, the process local staff access 
to participate in Chapter 2-funded T&PD 
activities and the interaction between local 
and state Chapter 2 offices. 

This visual model aided development of a 
more comprehensive questionnaire on the 
administration of local Chapter 2 T&PD 
programs. 

The Local Chapter 2 Evaluation Survey 
targeted school and district staff who 
actually participated in T&PD activities. 
The survey instrument was conceptualized 
through a series of meetings with ADE 
Chapter 2 staff and discussions with 
district Chapter 2 coordinators. 

In addition, ADE Chapter 2 staff shared 
their substantial base of knowledge and 
supportive documentation on the type and 
content of local T&PD activities. Utilizing 
t.Vrifl combination of information sources, 
R&D developed the preliminary version of 
the sur trey instrument. 

Prior to developing the Survey ofConference 
Effectiveness, R&D, Chapter 2 and ASAP 
staff discussed and exchanged information 
concerning structure, presentation and 
desired outcomes of the state-sponsored 
ASAP regional workshops which were held 
during spring 1991. 

Prom these discussions, general topic areas 
were identified, followed by articulation of 
the specific items that participants would 
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Field reviews 
of survey 
instruments 
often precipi- 
tated changes 
in length, 
presentation 
and content of 
the instru- 
ments. 



be asked in the survey instrument. Finally, 
construction of the ASAP Liaison 
Questionnaire was preceded by a series of 
preliminary interviews with seven of the 
1 6 targeted ADE staff persons. Information 
received from these interviews aided the 
development of general topic headings and 
wording of specific items included in the 
actual survey instrument. 

Second, preliminary drafts of the survey 
instruments were developed. R&D and 
Chapter 2 staff worked together to ensure 
integrity of content, coverage and 
quantitative validity of all survey 
instruments. The preliminary versions of 
these instruments were constructed after 
discussion and research among ADE staff 
and interactions with the target audience. 

A great deal of work went into developing 
the presentations of the Local Chapter 2 
Evaluation Survey and the District 
Coordinator Survey. Because these two 
surveys served as the primary data 
collection instruments from which to 
evaluate the effectiveness of Chapter 2- 
funded T&PD activities at the local level, it 
was felt that extra effort should be made to 
logically format and present each survey 
form. 

As a result, both instruments went through 
a number of formatting and typesetting 
revisions until an optimum balance between 
the final product and its ability to collect 
the required information was obtained. 
Each instrument included definitions of 
terms, descriptions of sections and 
explanations of intent. Hopefully, the 
additional time and effort spent in this 
regard resulted in higher return rates and 
improved data quality. 

Third, field reviews of the draft instru- 
ments were conducted. For the local 
evaluation and district coordinator 
instruments, preliminary forms were sent 
to district coordinators for review and 
comment. 



Based on responses, changes were made to 
the length, presentation and content of the 
instruments. For the conference 
effectiveness instrument, interviews and 
reviews of the preliminary instrument were 
conducted by ADE ASAP staff. These 
individuals were the primary facilitators of 
the conference and workshop sessions 
being evaluated and also acted as liaisons 
to the district personnel who attended the 
sessions. 

Finally, since construction of the interview 
questions used in the ASAP Liaison 
Questionnaire resulted from the one-on- 
one preliminary interviews with these 
individuals no additional field testing was 
conducted. 

Review of the preliminary instrument was 
performed collectively by R&D and Chapter 
2 staff in regard to content, scope and 
suitability to the overall evaluation of 
Chapter 2-funded T&PD activities. 

Fourth, instructions for the administration 
of the instruments were developed. For this 
activity, the process linked to each 
instrument included the designation of 
individuals responsible for disseminating, 
collecting and answering questions from 
the target group 

For the Local Chapter 2 Evaluation Survey, 
district T&PD coordinators were sent a 
complete list of all individuals in their 
district targeted to fill out a survey. 
Coordinators used this list to identify all 
persons receiving and returning a completed 
questionnaire as well as to follow up on 
individuals that or had not finished the 
task This process facilitated a higher than 
expected return rate of 89 percent 
throughout the 43 districts within the study 
group. 

For the District Coordinator Questionnaire, 
staff of the R&D and Chapter 2 units 
developed detailed instructions for 
completion and return of the survey. In 
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addition, check lists were maintained at 
the ADE to identify coordinators who had 
not returned a completed instrument. 
Follow up on all late or incomplete surveys 
also was undertaken. This resulted in usable 
responses from 41 of the 43 districts in the 
study group. 

For the Survey of Conference Effectiveness, 
ADE staff serving as facilitators of the 
ASAP workshops were provided copies of 
the instrument and instructed on survey 
implementation. Participants in the 
workshops were asked to fill out the survey 
and return the completed form before they 
departed. 

The facilitators were instructed to answer 
questions and assist participants with the 
completion of the instrument Having the 
ADE facilitators present to assist partici- 
pants with the taskincreased the estimated 
response rate across all of the workshops 
was 75 percent. 

Unfortunately, an exact count of 
participants in each workshop was not 
available. The response rate estimates used 
here was developed from interviews with 
ADE workshop facilitators. 

Fifth, after the survey instruments were 
administered to the target groups, the 
completed instruments were processed. 
With the exception of the ASAP Liaison 
Questionnaire, all of the completed survey 
instruments were numbered sequentially 
before being entered into the R&D computer 
systems. 

This provided a unique number for each 
survey and served as the primary record 
locator once the computerized databases 
were developed. By utilizing this number, 
errors in data input or questions regarding 
specific responses could be tracked easily 
back to the original survey instrument 

After the surveys were numbered, response 
codes indicating the answer to each survey 



question were placed in the right hand 
margin of the survey form. The response 
codes were denoted in red ink so that they 
would stand out at time of input. This 
helped to reduce data input errors and to 
make it easier for support staff to keep 
track of their progress. Prior to inputting 
the actual survey data, R&D researchers 
created preformatted data input forms to 
assist support staff with entering the 
information. These forms matched the 
order, format and type of questions on the 
original survey form. 

Along with the response codes noted directly 
on survey instruments, these predefined 
input forms improved the speed and 
efficiency of the data input process and 
helped minimize the error rates associated 
with processing the instruments. 

All of the statistical analysis for this project 
was performed within the Statistical 
Package for the Social Sciences (SPSS) 
software application program. Prior to 
developing any of the databases, R&D staff 
developed and tested the SPSS system files 
and programming logic using fabricated 
data for each instrument. 

In addition, output formats also were 
developed and tested to ensure that the 
information required from each instrument 
was properly presented. After the data for 
each survey had been computerized, 
frequency tables were run on every variable 
to ch eck for data entry errors or suspicious 
responses. 

R&D staff were able to investigate all 
problem data points by use of the record 
locator number assigned to both the 
completed survey instrument and its 
corresponding computer record. This cross- 
checking activity helped to minimize data 
entry errors and to highlight any problem 
responses. Finally, all of the calculations 
and data utilized in the evaluation report 
were double-checked for accuracy and 
consistency prior to publication. 4 
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ARIZONA DEPARTMENT OF EDUCATION 

Research & Development Division 
1535 W. Jefferson 
Phoenix, Arizona 85007 



SURVEY OF CONFERENCE EFFECTIVENESS 

- Arizona Student Assessment Program - 



Purpose: ASAP conferences were scheduled for Fall 1990 and Spring 1991. t ni ^SlSS^ 
P assistance given to K-12 educators in the local design, implementation «fvd4J^^n^»C« of 
the Arizona Student Assessment Program (ASAP). \ / ,/ 

Your responses to the following questionnaire will assist the Arizona Depa^e^ ^catton 
(ADE) in evaluating the effectiveness of the ASAP conferences. The resulfs^of thte evaluation 
will help ADE determine the type and amount of assistance to be provided ire|he future. 

Oroanlzatlon- Thi* questionnaire is comprised of three -parts. XparN requests information on 
Organization, jues o ^ educational -attainment and basic job descnptors. 

Part2 asks for feedback on the delivery of todays ASAP conference. Finally, rf you were 
in attendance, Part 3 requests feedback on the. Fall 1990 ASAP conference which initially 
introduced the Arizona Student Assessment; Program. 



Each person participating In today's 
conference on the implementation of the 
Arizona Student Assessment Program should 
complete this hsurtty/'. 

Please begin- by answering questions which 
help distinguish you In terms of personal and 
professional characteristics. 

Note that personally-Identifiable Information 
such as your name or social security number 
Is not requested. 

You are encouraged to respond In an open 
and frank manner. Completion of this 
questionnaire is expected to take 7 minutes. 



1. 
2. 
3. 



Today's date: 

District of employment: 

Location of this training session: 



- Part 1 - 
Respondent Demographics 



A. Participant Characteristics 

1 . Please indicate your gender. Check (✓) one: 

2. What is your race/ethnicity? Check (✓) one: 



□ Female □ Male 

□ Asian/Pacific Islander 

□ Black /'/'"X"' 

□ Hispanic / ' \ % 

□ Native American/Alaskan Native. 

□ White 



B. Educational Attainment 

What is your highest level of educational attainment? Please check (✓) one:. 

□ High School Diploma □ Associate Degree 

□ Bachelor's Degree □ Master's 'Degree 

□ Doctorate El "Post-doctorakStudy 

□ Other (Please specify): : 

Did you receive your highest degree in a field relatert to education, such as Elementary or 
Secondary Education or Master's in Publi&Schooi Administration? Check (✓) one: 

□ Yes □ No 

C. Job Descriptors ''^^V'' 

5. What is your primary Job classification at this time? Please check (✓) one: 

□ Administrator □ School Principal/Assistant Principal 
□..Classroom TSaetier : □ Librarian 

Q Teacher Aide... □ Counselor or Other Pupil Services Staff 

□ Other ,(PIease:specify): 

6. How long have you worked at your present position? Check (✓) one: 

□ Less than one year □ Five years, but less than 10 

□ One year, but less than 3 □ Ten years, but less than 15 

□ Three years, but less than 5 □ Fifteen or more years 

7. Are you a member of a site-based school improvement team? Check (✓) one: 

□ Yes □ No 

8. Do you participate in the planning of school-wide improvements at either the district or local level? 
Check (✓) one: * 

□ Yes □ No 




- Part 2 - 
Delivery of Today's ASAP Training 

In parts 2 and 3 you are asked for your impressions of the assistance you have been given. You are to code 
your responses to certain qualitative statements being made on the pages which follow. This portion of the 
questionnaire asks for the level of your agreement or disagreement with these statements. 

Circle the number which best describes your response based upon the range provided: 
(SA) (A) (D) (SD) 

Strongly Agree Agree Disagree Strongly Disagree 

1 2 3 /***4 



A. Facilitators (ADE Staff Assisting with Confsrsncs Sessions) S A '^^^JL| S D 

1. Facilitators were well prepared to conduct this session. ^ 1 \ 2 v / 4 

2. The goals and objectives of toda/s session were clearly / f r\ *h / 
communicated. / \ 1 \ 3 

3 . Answers to the questions that were asked in both the small jroufK, J / : 
sessions and conference wrap-up helped me to better^nderstand % 

issues that were unclear. / ,. V' 2 3 4 

4 . Facilitators did a good job of supporting and coordinating the activities %/' 

and discussions hi the small group sessions. s : ^ :: \ v ' /' 1 2 3 4 

5 . My understanding of ASAP, the goals and dbj^ctives of. the prqg^m, 
and my responsibilities in terms of future tasks h^ve beer> enhanced 

as a result of the facilitators' efforts. \ % t \ | / 12 3 4 

B. Handouts and Presentation Mat^aJs § A _C_ -SJL 



6. The handouts and presentation materials, werrclearty written and 
useful in helping me to understand the taskr! am expected to 
accomplish in the future* 

7. These materials will be useful as reference tools for implementing 
ASAP acuities. % , 

8. These ^atSWa^wiHbe usefui in informing others in my district/school 
about ASAF^activrties., / / 



12 3 4 
12 3 4 
12 3 4 



C. Small Grdup Sessions 5 A -0- 

9. The "hands-on" tasks performed in the small group sessions 
enhanced my ability to understand and plan for ASAP activities in niy 

district. 12 3 4 

10. My small group session provided an opportunity for me to clarify issues 

and answered questions I had on ASAP tasks and activities. 12 3 4 

11 . My small group session was conducive to open discussions and 
problem-solving with other participants and the facilitator nn ASAP 

tasks and activities. 1 2 3 4 

12. My small group session utilized an effective format ("hands-on" tasks 
and discussions) which enabled me to acquire needed skills and 

understand the information presented. 1 2 3 4 



Part 2 (Continued) 



D . Conference (Overall) § A A _D_ _§_&_ 

13. The conference was well organized. 12 3 4 

14. Attending the conference increased my understanding of ASAP 
goals and objectives in preparation for future ASAP activities in my 

district/school. 1 2 3 4 

15. The format of the conference provided an effective environment for / / *^w 

learning more about ASAP and clarifying difficult issues. 1 / ,,2w 3 4 



- Part 3 
October 1990 ASAP- Conference 



Did you attend the October 199Q ASAP Conference? \ Check (✓) one: □ Yes □ No 

It you answered "yes" to this question, please re&ppnd Tto. the following statements, it you answered "no," 
you have completed this evaluation. Please giv£ y£ur completed questionnaire to a facilitator prior to leaving 
the conference site. \ \ / 

Pall ASAP Conference S A A P -3JL 

1 . The Fall 1 990 ASAR. Conference .providdd : a "clear description of the 
goals and objectives^ ifmnew Arteofi&Student Assessment 

Program. \ \ y / ' 2 3 4 

2 . The Fall 1:990. ASAP Conference ctearly described the tasks and 
offort^istttel&iArilj.be requiredlo perform in implementing the new 

program. ' 1 2 3 4 

3. FadJitatSrs/presentersrat the Fall 1990 ASAP Conference answered 

most. of my, questions concerning the new program. 1 2 3 4 

4. As a result the the Fall 1990 ASAP Conference, I clearly understood 
how the new,assessment of students differs from current methods 

used in Arizona^ 1 2 3 4 

5. The Fall 1990 ASAP Conference was very useful in preparing me for 
today's regional meeting on implementing the new Arizona Student 

Assessment Program. 1 2 3 4 



Descriptive Statistics 



Part 1. Respondent Demographics (N=388) 



A. Participant Characteristics 

1. Please indicate your gender. 

Female 
Male 

Unusable/No Response 
Total 

2. What is your race/ethnicity? 

Asian/Pacific Islander 

Black 

Hispanic 

Native American/ 
Alaskan Native 
White 

Unusable/No Response 
Total 



Response 

232 
147 



388 



Response 

4 
2 
21 

9 

342 

_ia 

388 



Percent 
61.2 
38.8 



Perc 

1.1 
0.5 
5.6 

2.4 
90.5 



B. Educational Attainment 

3. What is your highest level of educational attainment? 



High School Diploma 
Associate Degree 
Bachelor's Degree 
Master's Degree 
Doctorate 

Post-Doctoral Study 
Other 

Unusable/No Response 
Total 



Response 

3 
0 

91 
233 

32 
6 

18 

388 



Percent 

0.8 

0.0 
23.8 
60.8 

8.4 

1.6 

4.7 



4. Did you receive your highest degree in a field related to education, such as elementary or 
Secondary Education or Master's in Public School Administration? 



Yes 
No 

Unusable/No Response 
Total 



Response 

331 
44 

12 

388 



Percent 

88.3 
11.7 
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Descriptive Statistics 



Part 1. Respondent Demographics (N=388) (continued) 



C. Job Descriptors 



5. What is your primary job classification at this time? 



Administrator 

School Principal/Asst. Principal 
Classroom Teacher 
librarian 
Teacher Aide 
Counselor or Other Pupil 

Services Staff 
Other 

Unusable/No Response 
Total 



Response 

103 
54 

167 
5 
2 

12 
33 
12 
388 



6. How long nave you worked at your present position? 



Less than 1 year 
1 year, but less than 3 
3 years, but less than 5 
5 years, but less than 10 
10 years, but less than 15 
15 or more years 
Total 



Response 

52 
82 
56 
85 
47 

s& 

388 



Percent 

27.4 
14.4 
44.4 

1.3 

0.5 

3.2 
8.8 



Percent 
13.4 
21.1 
14.4 
21.9 
12.1 
17 



7. Are you a member of a site-based school improvement team? 



Yes 
No 

Unusable/No Response 
Total 



Response Percent 

166 43.3 

217 56.7 

388 



8. Do you participate in the planning of school-wide improvements at either the district or local 
level? 
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Yes 
No 

Unusable/No Response 
Total 



Response Percent 

339 88.5 
44 11.5 

& 
388 
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Descriptive Statistics 

Part 2. Delivery of Todays ASAP Training 





Strongly 
Agree 


Agree 


Strongly 
Disagree 


Disagree 




# 


% 


* 


% 


# % 


# 


% 


A. Facilitators (ADE Staff Assisting with 
Conference Sessions) 
















1. Facilitators were well prepared to conduct 
this session. 


161 


42.7% 


188 


49.9% 


024 6.4% 


4 


1.1% 


2. The goals and objectives of today's session 
were clearly communicated. 


130 


34.6% 


213 


56.6% 


24 6.4% 


9 


2.4% 


*3 An qwpm tn the Questions that were asked 
in both the small group sessions and 
conference wrap-up helped me to better 
understand issues that were unclear. 


108 


23.7% 


211 


56.1% 


AO. 1 O QG/L 


9 


2.4% 


4. Facilitators did a good job of supporting 
and coordinating the activities and dis- 
cussions in the small group sessions. 


129 


34.6% 


195 


52.3% 


44 11.8% 


5 


1.3% 


5. My understanding of ASAP, the goals and 
objectives of the program, and my respon- 
sibilities in terms of future tasks have 
been enhanced aa a result of the facilita- 
tor's efforts. 


119 


31.3% 


217 


57.1% 


37 9.7% 


7 


1.8% 


B. Handouts and Presentation Materials 
















6. The handouts and presentation materials 
were clearly written and useful in helping 
me to understand the tasks I am expected 
to accomplish in the future. 


163 


42.9% 


196 


51.6% 


19 5.0% 


2 


0.5% 


7. These materials will be useful as 
reference materials for implementing 
ASAP activities. 


170 


44.9% 


185 


48.8% 


22 5.8% 


2 


0.5% 


8. These materials will be useful in 
informing others in my district/school 
about ASAP activities. 


149 


39.7% 


195 


52.0% 


28 7.5% 


3 


0.8% 
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Descriptive Statistics 

Part 2. Delivery of Today's ASAP Training (continued) 





Strongly 
Agree 


Agree 


Strongly 
Disagree 


1 

Disagree 




# % 


* 


% 


# % 


* 


% 


C. Small Group Sessions 














9. The "hands-on" tasks performed in the 
small group sessions enhanced my ability 
to understand and plan for ASAP 
activities in my district. 


119 32.2% 


193 


52.3% 


49 13.3% 


8 


2.2% 


10. My small group session provided an 
nnnnrfcunitv for me to clarifv issues and 
answered questions I had on ASAP tasks 
and activities. 


104 28.3% 


211 


57.5% 


50 13.6% 


i 


U.uvo 


11. My small group session was conducive 
to open discussions and problem-solving 
with other participants and the facilitator 
on ASAP tasks and activities. 


129 34.6% 


207 


re tz frf 

55.5% 


35 9.4% 


o 


U.Ovo 


12. My small group sessions utilized an 
effective format ("hands-on" tasks and 
discussions) which enabled me to acquire 
needed skills and understand the 
information presented. 


100 27.0% 


212 


57.3% 


53 14.3% 


5 


1.4% 


D. Conference (Overall) 














13. The conference was well organized. 


135 36.1% 


205 


54.8% 


31 8.3% 


3 


0.8% 


14. Attending the conference increased my 
understanding of ASAP goals and 
objectives in preparation for future ASAP 
activities in my district/school. 


136 36.2% 


199 


52.9% 


39 10.4% 


2 


0.5% 


15. The format of the conference provided 
an effective environment for learning 
more about ASAP and clarifying 
difficult issues. 


106 28.6% 


216 


58.4% 


39 10.5% 


9 


2.4% 
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Descriptive Statistics 



PartS. October 1990 ASAP Conference 



Did you attend the October 1990 ASAP Conference? 



Yes 
No 

Unusable/No Response 
Total 



Response 
211 
125 

388 



Percent 

62.8 
37.2 



If you answered "Yes* to this question, please respond to the following statements. 



Fall ASAP Conference 

1. The Fall 1990 ASAP Conference provided 

a clear description of the goals and objectives 
of the new Arizona Student Assessment 
Program. 

2. The Fall 1990 ASAP Conference clearly 
described the tasks and efforts districts will 
be required to perform in implementing the 
new program. 

3. Facilitators/presenters at the Fall 1990 ASAP 
Conference answered most of my questions 
concerning the new program. 

4. As a result of the Fall 1990 ASAP Conference, 
I clearly understood how the new assessment 
of students differs from the current methods 
used in Arizona. 

5. The Fall 1990 ASAP Conference was very 
useful in preparing me for today's regional 
meeting on implementing the new Arizona 
Student Assessment Program. 



Strongly 
Agree 

# % 


Agree 
# % 


Strongly 
- Disagree 

# % 


Disagree 
# % 


65 27.1% 


134 55.8% 


36 15.0% 


5 2.1% 


44 18.5% 


130 54.6% 


59 24.8% 


5 2.1% 


41 17.4% 


125 53.2% 


56 23.8% 


13 5.5% 


73 30.7% 


131 55.0% 


27 11.3% 


7 2.9% 


| 71 30.0% 


124 52.3% 


36 15.2% 


6 2.5% 
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Summary op Statistical Findings 

Part 1. Respondent Demographics 

• 60 percent female 

• 90 percent white 

• Educational Attainment: 

74 percent hold a bachelor's or master's degree 
10 percent hold Doctorates 

• 88 percent hold a degree in an education-related field 

• Primary Job Classification: 

44 percent Classroom Teachers 
27 percent Administrators 
14 percent School Principals 

• Years in Current Position: 

13 percent less than 1 year 

36 percent 1 to 5 years 

22 percent between 5 and 10 years 

29 percent more than 10 years 

• Involvement in Site-based/School-wide Improvement Activities: 

57 percent are NOT part of a site-based school improvement team 
89 percent DO participate in school-wide improvement activities 

• Fall 1990 ASAP Conference Attendance: 

63 percent also attended the Fall 1990 ASAP Conference 
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Summary op Statistical Findings 



Part 2. Delivery of Today's ASAP Training (continued) 



D. Workshop (Overall) 

(Survey Questions: Part 2.D13 - D15) 

Domain: Questions focused on how well the workshops were organized, the suitability of the 
format used to present and discuss ASAP issues and the workshop's usefulness in 
increasing understanding of ASAP goals, objectives and future district activities. 



Strongly 
Agree/Disagree Agree/Disagree 



Combined 



Positive 
Negative 



55.4% 
9.7% 



33.6% 
1.2% 



89.0% 
10.9% 
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Summary of Statistical Findings 

Parts. October 1990 ASAP Conference 



A. ASAP Information 

(Survey Questions: Part 3.1, 3.2, 3.4) 

Domain- Questions focused on how effectively the fell 1990 ASAP Conference provided a clear 
understanding of ASAP activities including goals and objectives, the tasks required by 
districts to implement the program and how ASAP departs from the traditional system of 
assessing student performance. 

Strongly 

Agree/Disagree Agree/Disagree Combined 
Positive 55.1% 25.4% 80.5% 



Negative 



17.0% 2.4% 19.4% 



B. Facilitators 

(Survey yuesttonb. Part 3.3) 

Domain: Question focused on the ability of the facilitator to answer questions on ASAP. 

Strongly 

Agree/Disagree Agree/Disagree Combined 

Positive 53.2% 17.4% 70.6% 

Negative 23.8% 5.5% 29.3% 

C. Preparation for Spring 1990 Regional Workshops 

(Survey Questions: Part 3.5) 

Domain: Question focused on the usefulness of the Fall 1990 ASAP Conference to prepare 
districts for the spring 1991 regional workshops. 

Strongly 

Agree/Disagree Agree/Disagree Combined 

Positive 52.3% 30.0% 82.3% 

Negative 15.2% 2.5% 17.7% 
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Questionnaire 
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ARIZONA DEPARTMENT OF EDUCATION 

Research & Development Division 
1535 W. Jefferson 
Phoenix, Arizona 85007 



LIAISON QUESTIONNAIRE 

Arizona Student Assessment Program 



A. Overall ASAP Training and Development 

1 . What do you feel your "role" is in the ASAP Program? 
Context Extensions: 



Within the ADE organization? 
Within the SEA/LEA relationship? 



Response: 



Let's talk about the process of learning that you went . through when firSUsifgrjed to ASAP activities. 

• Was the process clearly laid out and followed? ' 

• Did you have a sense of what you needed to le^rrt and Sow long it would take to learn it? 

Context Extensions: Organization? / .. '-.J 

Clearly defin^a^ehda? ' ,/ 
Goals and jefbjeptives ? . 
Principl&formatvf inservice sessions? 
Outside bprqultahts? ) /' 

Discussion: Format of the staff d&ve^pmentpfOeeh June 1990 - November 1990, prior to fall 
conference? """"VA \ \ 



Response: 



3 . Did the preparatiofvtrai'r^ng,you received on the ASAP program adequately prepare you for the 
actual activities you : :ha?ve.beeh:perf6rming? 

Context Extensions; ^Preparation for fall and spring conferences? 
/ ^*^v. x ,.. ' /Questions from districts? 

/ f .,, \ / Had a clear vision of what ASAP is and how it impacts districts/state? 
\ K ^ ,,>«.*. : ; ' How ASAP compares/differs from current assessment process? 
\ y X y ' ) } Quality of instruction/learning process? 

R e s p o n s r ,w ^ Tl 



4 How much input did you have in the development of the ASAP program? Were you involved in any 
planning sessions or review of materials/assessments or consulted about strategic directions? 



Context Extensions: 



Degree of group effort vs. purely support position? 
Sense of belonging/ownership to the ASAP effort? 
Responsibility for success/failure? 

Role/influence of ASAP Director and Associate Superintendents for 
ASAP Development? 



Response: 
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B. Spring 1991 Regional Workshops 



1 . What was your role in the regional workshops? 

Context Extensions: Your personal perspective — 

As an ADE staff person? 
As curriculum specialist? 
As a facilitator of small group discussions? 
As a presenter? 



Response: 



2. Overall, do you feel you were well prepared to conduct the training wori<srjop%atlhWegio^ 
conferences? \/' "\ f \ 

Discussion: Preparation for ADE staff development sessions and ths&tyice tfajnthgby^SAP 
staff ? : W 



Response: 



3. Before the workshops, did you have a clear sense of wfiat yocir respcwsibitffies would be? 

Context Extensions: Goals and objectives of thejworkshop? 

Your personal gd^is and objectives? 

What wereJh&poals artf objectives of the workshops? 

Response : 'i ^ 



4. Did you feel that you were suc^ssfu] the primary goals and objectives of the 
workshop to participants? 

Response : '. ; . •' 



5. DidyouJeelthat you were Successful in meeting these goals and objectives including those you set 
for ypursell? : * 

R e s p o rise : . 



6. Do you think that the participants left the workshops with a clear understanding of the purpose and 
activities of ASAP? 

Response : „ 



7. Do you feel that the workshops helped to inform districts of their responsibilities under ASAP? 
Discussion: Clarified issues, encouraged open discussions, exchanged information? 



B. Spring 1991 Regional Workshops (Continued) 

8. Did you use handout materials? 

Context Extensions: Do you feel that they were effective training tools? 

Did you have input into their development? 

Would they serve as useful reference materials back in the district 

offices? 

Response: - 



9 . Did you conduct M hands on" activities in the workshops? /V^C 

Context Extensions: Describe the activities/purpose. 

Did you help develop the activities? -I,.,,-,, * ^ ; 
Were they effective for what they were intended.to teach? / 

Discussion: District Assessment Plans, ASAP /\ \ 

Rpspnnfie: ^ " \ 



1 0. Were there a lot of questions and discussions during the workshop? • 

Context Extensions: Were they constructive, irtfdrrmtive and valuable to the participants? 

Do you feel that you had a good dialogue with the participants? 



1 1 . Were you well prepared to '"arii^ef^^uestiot^.aSked of you by the participants? 
Response: " ' " ; : " 



1 2. Would you. say that toe workshops were well organized? 
R e s p <? n s a*™*/ 



13. How woutd : y6u : ^ate the overall success of the workshops (Very High, High, Low, Very Low)^ 
Response: _ ;" ~~ 



ERLC 



ASAP Fall 1990 Conference 

1 . Did you participate in the planning of the Fall ASAP 1990 Conference? 

Discussion: Involvement in the process or involvement in group exchange as 
tool for ASAP training? 

Response: 



2. Did you participate in the Fall 1990 ASAP Conference? If Yes, please discuss: 
Response : 



3. What was your role in the fall conference? ., \ 1 \ 

Context Extensions: As a facilitator of small group discussions? ' • .. .£ w *^' / j: 

As a presenter of a particular aspect of the ASAP program? / y 
As a speaker? / 
Other? \ 

Respo nse : : ; •■ v ' 



4. Did you feel that the training you received on ASAP preparediyou for your duties at the fall 1990 
co nf e re n c e ? ' V 

Response: „ ^ 



5 . What do feel were the goals and objective^ the -tell , inference? 
Response: ^ \ 

6. Do you feel that the.fall 1990 conf^nee w^s : well organized and efficiently run? 
Response: ■■ , f - 



7. Do ypii feet that the fall 1990 conference provided participants with a clear description of the ASAP 
program afxThoyy it differs from previous testing processes? 

Response-: ''_ z - ,■ . 



8. Do you think roost participants left with a clear understanding of these differences? 
Response: , 



9. Do you feel that the fall 1990 conference provided a clear understanding of what districts would be 
required to undertake to implement ASAP? 

Discussion : Comparison with spring regional activities? 

Introduction to spring/summer ASAP activities? 

Response: 

1 0. Do you feel that the fall 1990 conference was a success in terms of its original goals and objectives? 
Response: 



Summary of Responses 



A. Overall ASAP Training and Development 
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1. What do you feel your "role" is in the ASAP Program? 

Context Extensions: Within the ADE organization ? 

Within the SEA/ LEA relationship? 

To contribute to the changing nature of instruction upon which ASAP is based. Support development of new 
^^SS^ZSSSSm Peaces. Encourage and motivate LEAa to support these new teaching and 
learning methods. 

ReviewformABandCandhelpdeve^ 
Sn^Wri^ 

C ontent area soecialists also help districts with curriculum developmentissues conforming to AbAJr . 1 hatis, giving 
^^SSSSS^y methods and content issues. ADE ASAP liaisons also heljped «f e p'actace 
TcorlnT sessions and with determining district attitudes on the non-test indictors to be included in the A.AP 
program. 

To provide information about program to the districts. This is a very important role. 

To keep schools informed on what the ADE is doing during the ongoing development of the ASAP program. 

Two levels of roles: First, tho content area specialists helped to develop the three assessments 

on content and learning strategies in the subject areas. Second, and everyone was involved in this, to help districts 

understand and implement activities under the ASAP program. 

2 Let's talk about the process of learning that you went through when first assigned to ASAP 
a^tiv^ wTthe process clearly laid out and followed? Did you have a sense of what you 
needed to learn and how long it would take to learn it? 

Context Extensions: Organization? 

Clearly defined agenda? 

Goals and objectives? 

Principle format of in-service sessions? 

Outside consultants? 

Discussion: Format of the staff development process, June 1990 ■ November 1990, prior to 
fall conference? 

TheADEconductedalargenumterofinternal^ 

b^theADE^ 

staff needed to know. 

The in-services involved learning about the basic premises and tenets of ASAP and then reading and discussing 
among ADE staff the many unanswered questions that still needed to be worked out 

Par West Labs were brought in to help facilitate ADE in-services on two occasions. They guided discussions on 
LfinYng^ what the primary purpose was and what they needed to accomplish. 
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Summary of Responses 



2* (continued) 

Monthly meetings were held. During these meetings, staff went through all of the essential skills, the assessment 
scoring process, curriculum alignment, sessions on how to give ASAP presentations, how the rubrics to the 
assessments were developed, the assessments themselves and the process of developing anchor papers. There were 
a lot of discussions and questions. We practiced scoring the assessments and discussed differences in results. The 
entire process was well organized. The structure ' Jlayed a lot of fears about the new ASAP program. 

The basic organization was very good. There were differences of opinion, but the process got us through and provided 
answers to most of the questions. Everyone felt strongly about ASAP. Some did not like the "canned" discussions 
we were expected to learn. 

The process was not clearly defined — no clear direction nor was there a clearly defined goal. The philosophy of 
performance-based assessments (PBA) was there but we didn't know how to get there at the beginning. We were 
all learning. Had to turn to many papers and research to help with the learning process. Relied on a lot of coaching 
from the ASAP director. This was not necessarily bad in that training on ASAP was both a process of learning and 
a process of creation. 

Learning ASAP was a process. At first, the ASAP director provided the in-services and knowledge about the 
program. However, each ADE staff person had to spend a lot of time reading and discussing with colleagues about 
what it was all about 

At the beginning, the process of learning about ASAP was frustrating. It was not well defined. The global context 
was thought out, but not the details. But this was due the simultaneous implementation and creation of the ASAP 
program. 

The ASAP director provided almost all of the learning materials. ADE staff learned their roles by listening and 
reading about ASAP and similar programs. Many details were developed as we went along. Goals and objectives 
were defined/refined as we went along. 
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3. Did the preparation/training you received on the ASAP program adequately prepare you 
for the actual activities you have been performing? 

Context Extensions: Preparation for fall and spring conferences ? 

Questions from districts? 

Had a clear vision of what ASAP is and how it impacts districts I state ? 
How ASAP compares /differs from current assessment process? 
Quality of instruction /learning process? 

Yes. ASAP staff grew along with the evolution of the ASAP program. 

Yes, very well. I know for some it did not, but for me it worked well. The training gave a good clear understanding 
of ASAP. 

The sessions on assessment scoring were exceptional. 

Felt incredibly on my own. Had to constantly go back and talk to director and review research writings to gain new 
knowledge and new perspectives. However, this was necessary because of the new territory that was being covered. 

ASAP moved too fast. It was not well thought through. Districts felt uneasy with the speed of its implementations 
and timetable. Not being involved in more of the planning aspects, aggravated this feeling. Felt that all of the 
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3. (continued) 

information on the ASAP program (status and problems) was not being passed along to the ADE staff and to the 
districts. 

The training materials used by the ADE were also very useful in interacting with the districts. This helped in 
preparation and knowledge. Peer-to-peer training helped greatly. 

The structured in-service sessions were less important than the peer-to-peer interaction. 

4. How much input did you have in the development of the ASAP program? Were you 
involved in any planning sessions or review of materials/assessments or consulted about 
strategic directions? 

Context Extensions: Degree of group effort vs. purely support position? 

Sense of belonging /ownership to the ASAP effort? 
Responsibility for success I failure? 

Role I influence of ASAP Director and Associate Superintendents in 
Development of the ASAP Program ? 

For the content-area specialists, input to the development of the assessments was important For others, input into 
the directions and policies of ASAP was very limited. The ASAP director determined most of the direction with the 
exception of questions used in the assessments. These were reviewed by the ADE content-area specialists. 

None whatsoever. None in PBAs. None in essential skills or the ASAP philosophy. However, some ADE staff did 
help with the development of the PBAs because it was their area of expertise. Lack of participation is not necessarily 
bad, h owever, because a committee process would have killed the whole development of ihe ASAP. Decisions needed 
to be made. This has not caused a lack of ownership toward the program. 

Moderate amount Participation in meetings and discussions really had an impact. The input from the districts 
really had an impact as well. 

Sometime the ASAP director would say something in big meetings or to districts that had not been told to the ASAP 
staff yet. We would find it out from the district people. This was embarrassing. It was a communication breakdown. 

Ninety percent ADE Administrators, 10 percent ADE staff. This was ihe balance of input to developing the ASAP 
program. We had to buy into the vision. We were told the vision instead of being asked about it. 

Not much. Directions were always set by the ASAP director. But we needed this type of leadership due to the 
complexity of the ASAP program. It could not have been done by committee. However, over 3,000 comments were 
received from the districts on the program. 

ADE staff were told they were active participants, but in reality they were not. Suggestions that were not in line 
with already developed ideas were not acted on. 

Initially, there was no input from the ADE staff. However, this has changed considerably because ASAP has 
changed. At the beginning, leadership was necessary to get things done. Now, ADE staff has considerable input into 
the program and its actual implementation. 

The organizational structure has changed. Before, ASAP was something someone else was developing. Now, ADE 
staff are collectively involved and there is a lot of information sharing and input 
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Summary op Responses 

B. Spring 1991 Regional Workshops 

1. What waa your role in the regional workshops? 

Context Extensions: Your personal perspective- 
's an ADE staff person? 
As curriculum specialist? 
As a facilitator of small group discussion? 
As a presenter? 

Facilitator of small group sessions. District representatives for. each ADE ASAP liaison attended regional 
workshops to learn about and practice preparing the ASAP District Assessment Plan (DAP). 

Presenting DAP materials and helping participants work through the examples. 

Presenter. Provided updates and information on ASAP activities. Answered questions on curriculum alignment. 
Explained how district could use the DAPs in their ASAP planning process. 

2. Overall, do you feel you were well prepared to conduct the training workshops at the 
regional conferences? 

Discussion: Preparation from ADE staff development sessions and in-service training by 
ASAP staff? 

No. Workload outside of ASAP duties prevented some of the ADE staff from preparing adequately for these sessions. 
However, the feeling was that they were generally well prepared for ASAP duties. 

Yes, generally well prepared to conduct the workshop. 

Didn't feel well prepared psychologically, but the materials helped a lot to get through the sessions. After the 
meeting was over, I felt very good about it. 

For the mostpart,yes. Butthisisfrom the standpo 
the ADE ASAP in-service training process. 

3. Before the workshops, did you have a clear sense of what your responsibilities would be? 

Context Extensions: Goals and objectives of the workshop ? 

Your own personal goals and objectives? 

What were the goals and objectives of the workshops? 

Yes. Responsibilities included assisting districts with the agenda and activities required to work tough the major 
ASAP assessment areas and to match the essential skills lists with the upcoming new assessments. Also, 
^sponsTbS included providing the opportunity for districts to discuss ASAP issues and to share expenences. 
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3. (continued) 

Yes, the ADE staff had an agenda to work from. 

The goals of the regional workshops were carefully scripted by the ASAP director. 

4. Did you feel that you were success ?ul at communicating the primary goals and objectives 
of the workshop to the participants? 

Yes. This was helped by having good discussions and answering questions. 

Small groups helped ensure that there was good communication. 

Yes. Very successful. 

5. Did you feel that you were successful at meeting these goals and objectives including those 
you set for yourself? 

All respondents said yes to this question. 

6* Do you think that the participants left the workshops with a clear understanding of the 
purpose and activities of ASAP? 

Yes. Our group did* They asked a lot of questions. Some left very confident while some left frustrated that other 
districts were ahead of them. Mainly, I found that the teachers were in support of ASAP but administration was 
not I believe that districts who rely on ITBS scores to score high achievement resist moving to ASAP for fear that 
they will not be able to show how well they are doing compared to other districts. 

Not totally clear, but more so had the workshops not been held. Some of the participants were being exposed to the 
ASAP program for the firist time. 

They left with a clearer understanding than when they came, but it wasn't a perfect understanding of ASAP. No 
one had this. 

Most left with a clear sense of what they had to do. 

7. Do you feel that the workshops helped to inform districts of their responsibilities under 
ASAP? 

Discussion: Clarified issues, encouraged open discussions, exchanged information? 

Yes. DAP was an important tool to introduce the districts into how to prepare and plan for implementing many of 
the ASAP activities. 

Yes. The workshops were the start of the DAP process where districts began to plan for implementation of ASAP. 
There ^vas no negative feedback. 
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Summary of Responses 

7. (continued) 

Yes. The DAP gave the districts the information they needed to take back to their local organizations. 

8. Did you use handout materials? 

Context Extensions: Do you feel that they were effective training tools ? 

Did you have input into their development? 

Would they serve as useful reference materials back in the district 
offices? 

All the regional workshops used handout materials designed by the ASAP director. 

The handouts were very effective in helping districts work through the conceptualization process in organizing 
ASAP activities. 

The handouts were developed by the ASAP director. Very little input was requested from ADE ASAP staff. 

Apart from the DAP worksheets, additional handouts were prepared independent of the original ones provided. 

The handouts provided the basic information needed to the participant. 7ne> bervcd ill* purpose very well The 
ADE staff did not have any input into the development of the handouts. The handouts would be very useful as 
reference materials back at the district offices. 

The handouts were not as effective as they could have been. There were too many little parte to the ; wortaheeta. The 
worksheets were too fragmented However, the other h^ 
tools and as reference materials. 

Neutral on the effectiveness of the handout materials. The real value of the workshop came from the interaction 
of participants and the facilitators in exchanging information and answering questions. 

The handouts were the exact forms that districts would be using for reporting DAPs back to the ADE. 

9. Did you conduct "hands on" activities in the workshops? 

Context Extensions: Describz the activities /purpose. 

Did you help develop the activities? 

Were they effective for what they were intended to teach? 

Discussion: District Assessment Plans, ASAP 

Purpose of the hands-on activity was to get the districts thinking about district assessment plans and what they 
needed to submit to the ADE. 

Districts went through a process of placing each of the essential skills in specific grades. This was the essence of 
fhe DAP-plning Tut the sequence of teaching the essential skills according to a developmental or earning 
procesT Discussions were had on when and where to best begin teaching certain essential skills and how to assess 
student mastery of them. 

er|c 10 110 



Summary of Responses 



9. (continued) 

Yes. The hands-on activities were a central part of the regional workshops and were very effective in exposing the 
districts to the ASAP planning and implementation process. 

The hands-on activities were effective. 

No. ASAP staff liaisons did not help develop the hands-on activities. They were provided by the ASAP director. Since 
the forms used in the hands-on activities were the exact ones that the districts would use to report their DAPs back 
to the ADE, the activity was very useful in instructing the districts about what was needed to be completed. 

The participants knew more about the DAP process as a result of hands-on activity. For this reason it was effective. 

10. Were there a lot of questions and discussions during the workshop? 

Context Extensions: Were they constructive, informative and valuable to the 

participants? 

Do you feel that you had a good dialogue vnth the participants? 

Yes. There were a* lot of questions and discussions among the district participants. Most questions were not 
negative. District were eager to learn'what was required under ASAP. The dialogue was very good among district 
participants. 

A lot of questions that were asked could not be answered yet 

No. There were not a lot of questions. However, information sharing through discussion was one of the main 
strengths of the workshop. 

Expected amount of questions. "Need to know* type questions. There was also good dialogue among participants. 

11. Were you well prepared to answer the questions asked of you by the participants? 

Yes. Generally well prepared to answer most questions. There were some questions that the liaisons could not 
answer because ASAP had not yet progressed far enough through the process. 

12. Would you say that the workshops were well organized? 

Yes. 

Planning could have been better for the workshops. 

13. How would you rate the overall success of the workshops (Very High, High, Low, Very 
Low)? 

Very high. Good. Moderately high. High; Medium. High. Acceptable. High. Very high. High, but not very High. 
Generally a success. • 
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C. ASAP Fall 1990 Conference 



1. Did you participate in the planning of the Fail ASAP 1990 Conference? 

Discussion: Involvement in the process or involvement in group exchange as tool for ASAP 
training? 

Not really. Most of the planning, materials and scripting of sessions was completed by the ASAP director. ADE 
ASAP staff had little input into the actual development This is similar to the Spring regional workshops. 

Helped with the site selection. 

Helped in the planning, facilitator of small group sessions and met with assigned regional districts. 
There was top-down involvement. 

2. Did you participate in the Fall 1990 ASAP Conference? 

Yes. All wf the ASA? ilstrict liaison staff and ASAP staff participated in the fall conference. 
If Yes: 

3. What was your role in the fall conference? 

Examples: As a facilitator of small group discussions? 

As a presenter of a particular aspect of the ASAP program? 

As a speaker? 

Other? 

Some staff were facilitators of small group sessions to discuss aspects of the new ASAP program. Others made 
presentations such as essential skills within math, reading and writing. 

Presenter at small group workshop in specific content area* 

Facilitator at regional small group sessions. Lead discussions on ASAP issues. Clarified issues. 
Presenter at sessions on curriculum alignment. 
Small group session on upcoming DAPs. 

4. Did you feel that the training you received on ASAP prepared you for your duties at the fall 
1990 conference? 

Did not feel as well prepared for the fall 1990 conference as with the spring 1991 regional workshops. Spring 
workshops gave more time to become knowledgeable about ASAP and its many components. 
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4. (continued) 



Felt more prepared for the fall conference than the spring regional workshops because the breakout sessions some 
staff lead concerned their own particular content area specialty. 

Did not feel confident going into the fall conference because ASAP had too many unanswered questions at the time. 
Yes, felt confident in the information and what was needed to be accomplished. 

5. What do feel were the goals end objectives of the fall conference? 

Alert districts of their ASAP responsibilities, district assessment plans and provide introduction to the ASAP in 
general. 

To find out what the next steps would be after discussing ASAP with the districts and finding out their degree of 
involvement. 

Introduction to PBA in general and ASAP in particular. Give as much information on ASAP as possible. 

Forum to bring together districts and ADE ASAP staff, introduce ASAP and begin planning process, preparations 
for spring. 

Give districts information on ASAP. 

Provide direction to districts in planning for ASAP. 

6. Do you feel that the fall 1990 conference was well organized and efficiently run? 

Not as well as it could have been. Problems occurred when large and small districts were put together in the same 
discussion groups. The large districts resisted the change to ASAP more than the smaller ones. This caused some 
friction in the discussions. Overall, the conference presented the essential features of the ASAP program but some 
of the more detailed small groups sessions could have been better organized. 

In general, yes, it was well organized. 

Yes, but it didn't meet the needs of everyone because they were at different stages of development. 
Yes, definitely. 

7. Do you feel that the fall 1990 conference provided participants with a clear description 
of the ASAP program and how it differs from previous testing processes? 

In general, yes. But some participants left not quite sure what the full implications were of the new program. 
Yes. 

Becausesomedistrictshadalreadybeeninvolved with ASAP, disparities existed between the districts. This caused 
some problems. Some were just starting while others were advancing. 
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8. Do you think most participants left with a clear understanding of these differences? 
Some did not from the standpoint of the details and implications of implementing ASAP. 

The differences were communicated, but they may not have fully grasped them by attending one two-day 
conference. However, it started the process of inquiry into ASAP. 

While not everyone fully understood the differences or what the implications of ASAP were, it was planned that the 
spring regional workshops would be held to clear up any questions or confusion through use of DAP practice 
sessions. 

Generally, yes. 

9. Do you feel that the fall 1990 conference provided a clear understanding of what districts 
would be required to undertake to implement ASAP? 

Discussion: Comparison with spring regional activities ? 

Introduction to spring I summer ASAP activities? 

Not completely. This was the zoic ox liie spring regional workshops. 

No, not clear. The general ideas were given, but explanation and discussion were needed 

10. Do you feel that the fall 1990 conference was a success in terms of its original goals and 
objectives? 

Yes. Rated as Good, but not real good. 
Overall, very good and very successful. 
I would rate it high, but not very high. 

The conference was of average success for what it intended to accomplish. 

It was good for what it was intended for, but not great. There were those who didn't completely understand or agree 
with the new ASAP and some didn't really comprehend what its implications were. 

The conference should receive a "high" rating. 
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ARIZONA DEPARTMENT OF EDUCATION 



Research & Development Division 
1535 W. Jefferson 
Phoenix, Arizona 85007 



LOCAL CHAPTER 2 EVALUATION SURVEY 

- District Programs of Training & Professional Development - 



Purpose: This survey examines strengths and weaknesses of your district's Training >^mfess^al 
Development Program. You and other Arizona educators, selected at jandom?:are : ask|di|o 
complete the following questionnaire. / „ I \ 

Your responses enable the Arizona Department of Education (ADE) to measure ^ecliveriess 
of the Chapter 2 Program for the school year 1990-91 . The U. S. Department^ Education-will 
share survey results with Congress upon publication and submission by ADE. \. / 

Oraanlzatlon: The questionnaire consists of three parts. Part 1 requests fnformatton on respondent 
demographics. Part 2 asks you to gauge the,bene<its\you derived from the vanous 
courses or activities in which you participated,. Part 3 asks for your overfill assessment of 
the district's training program, across a!! courses^ activities you<are evaluating today. 
NOTE- A separate Part 2 is to be submitted lor each Chapter4tfaining activity, up to a 
maximum of five courses, whiclvyou attended during the 1990-91 school year. 

Do Dfll complete this survey unless you are' an administrator; 
teacher, teacher aide, orxdttier noncleriaf staff working in an 
Instructional or support services capacity! 

Please begin 6y answering questions which help distinguish 
you in terms of personal and professional characteristics. 
You are. encouraged to Ifespond in an open and frank 
manner; Nbte that personally identifiable Information, such 
as your name or sociaksecurity number, is nst requested. 

Completion of tUis questionnaire should take approximately 
.. 14 minutes. Thank you. 



1. Today's date: 



2. Districtof employment: 



- Part 1 - 
Respondent Demographics 



Participant Characteristics 

1 . Please indicate your gender. □ a) Female □ b) Male 

2. What is your race/ethnicity? Cheekbone: □ a) Asian/Pacific Islander □ b) Black □ c) Hispanic 

□ d) Native American/Alaskan Native □ e) White 



3. School where you work: 



(NOTE- If you usually work at the district's administrative office, indicate "District Office" as your choice. If you 
& are assigned to more than one school, please indicate "Multiple Sites" as your choice.) 

mc a is 



Educational Attainment 



4. What is your highest level of educational attainment? Please check (✓) one: 

□ a) High School Diploma □ d) Master's Degree □ f) Post-Doctoral Study 

□ b) Associate Degree □ e) Doctorate □ g) Other : 

□ c) Bachelor's Degree 
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What is your primary job classification at this time? You may wish to refer to Appendix A: Definition of Terms in 
order to assist you in selecting the appropriate response. Check (✓) only one box, a t|v6 : ijgh4; : 



ADMINISTRATORS 

□ a) Superintendent/ Assistant 

Superintendent 

□ b) School Principal/ Assistant 

Principal 

□ c) Governing Board Member 

□ d) Other District Administrators 



INSTRUCTIONAL PERSONNEL 

□ e) Certified Teacher 

(If choosing this response, p/eas#' :: 
answer questions 3 through 13.) 

□ f) Teacher Aide 

(If choosing this response, please 
answer quest^nsV thrviigh 13.) 

□ g) Other instmctioMstaff \ 



NQNCLER^QAL SUPPOF 

dyjj| Libtari|n\ 
U j)^QoupSej0r 
U \ Othef 

r % . (Including psychologies 
■ : ittendance, health, ana 
other pupil services sla 



6. How long have you worked in your present position? Cheek one: 

□ a) Less than one year □ c) Three. years; .but less than 5 e) Ten years, but less than 1 

□ b) One year, but less than 3 □ d) Five years, but less than 10 □ f) Fifteen or more years 

7. Are you a member of a site-based schooiimprov^ent team? Q 'a) Yes □ b) No 

8. Do you participate in the planning prtmplententMon offiehodlwide improvements (i.e., the process of 
curriculum restmcturing orothereducatidnal.^ Q a) Yes □ b) No 



If you are a classroom l&qcher or a teacher aide, please complete questions 
9 through 73. Other respondents should leave these Items unanswered. 



9 . What is the average number of studertts '0t class to whom you provide instruction? 

(Teachers and Teacher Aides shciuJd total the number of students they work with on a regular school day and 
divide by the number of classes in vr&er to arrive at an average.) 

Average puw^^f jtuderts per Blass: 

1 0. Which grade leveLdescribes the majority of students with whom you interact daily? Check (✓) only one: 

□ a) : R.fesjpbooi □ b) Kindergarten □ 1 □ 2 □ 3 □ 4 □ 5 □ 6 □ 7 □ 8 

'f^/ □ 9 □ 10 □ 11 □ 12 

11. How many years in total have you worked in instruction? Check (✓) one: 

□ a) Less than one year □ c) Three years, but less than 5 □ e) Ten years, but less than 15 

□ b) One year, but less than 3 □ d) Five years, but less than 10 □ f) Fifteen or more years 

1 2. In which primary academic/vocational area do you deliver or assist with instruction? Check (✓) one: 

□ a) Preschool -6 (all subjects) □ d) Vocational Education □ g) Performing and Creative Arts 

□ b) Math and Science □ e) Foreign Languages □ h) Other (Specify:) 

□ c) Language Arts □ f) Humanities 



1 3. Are a majority (50% or more) of students in any of your classrooms considered to be "at risk?" Check (✓) one: 
□ a) Yes □ b) No □ c) Unknown 



| THIS IS THE END OF PART 1. NOW PROCEED TO PART27 



n 



- Part 2 - 



INDIVIDUAL COURSE EVALUATION FORM | 

Local Chapter 2 Coordinators have researched and identified all Chapter 2 training and staff development activities 
attended by their staff during the 1990-91 school year. You should possess sufficient copies of Part 2 in your packet 
to complete one assessment for each of a total of five (5) courses or training activities identified. 
Part 2 is a course-by-course overview of the professional and educational benefits you derived from various training or 
staff development activities. As respondents who understand assessment techniques, your perceptions are cntical! 
All questions are to be answered, unless otherwise indicated. Thank you again for your helD)"***v^ 



Course Title 

1 What is the Course Title which best describes the training or staff development activity :i ypu ajtended? Use 
Apppnriix R: I istinn of Courses and Train ing Components in completing this question. \y y 
If you are unable to find a reasonable match to your Descriptive Course- Title, please supply an alternate. 

Descriptive Course Title: _ -— — — ; couseNo) 

(From Appendix B) y""-""*^ ( uouisewo.) 



Alternate Title You Chose: 




2 . If you were required to identify an alternate Course Title, please Assist us reclassifying its general content area 
from the listing below. Check (✓) only one Liox;. * x , ' </ 

□ a) Instructional Delivery □ e) Performance Assessment^ □ i ) Schoolwide Improvements 

□ b) Academic Education □ f) Staff Effectiveness training □ j) School Environment 

□ c) Pupil Motivation Q,g) Instrucftqnal ^ead^rship □ k) Parental Involvement 

□ d) Behavioral Modification ,ti h) Cuajcular Services □ I) Other: 

Course Purpose, Focus, Format, and ^4<&a&Pi odations 

In answering questions 3 thrqugi H^you may wilb to feferto Appendix A; Definition of Terms to assist you in selecting 
the correct response. Check Hy\ only dnexbox per question. 

3. Which of the following^ategorfe&bestdescribes the purpose of the training course you attended? 

□ a) Employee Orientation 1/6 c) Inservice Training □ e) Staff Development 

□ bJ PreseTvi^Training ' ' /y y □ d) Retraining 

4. Which of these. categories best describes the focus of the training course you attended? 

□ a) Potkjfes & Prbcedures □ d) Administrative Skills Training 

□ b) Instmjionaf Practices (If you have checked Instructional Practices, complete question No. 5 below.) 

□ c) Non-Instructional Methods □ e) Supportive Services Skills Training 

5 . If the primary focus was Instructional Practices, check (✓) one box for each of the following inquiries: 

A. Designed to expand my knowledge of curriculum development? □ 1) Yes □ 2) No □ 3) Not sure 

B. Designed to assist me in the effective delivery of instruction? □ 1) Yes □ 2) No □ 3) Not sure 

C. Designed to offer state-of-the-art pupil assessment strategies? □ 1) Yes □ 2) No □ 3) Not sure 

6. Which ci these categories best describes the training format you experienced while attending this course? 

□ a) Structured Course Work (Credit or Noncredit) □ b) Conference □ c) Seminar □ d) Workshop 

7. Which of the these categories best describes the accommodations made for attendance at this course? 

□ a) In-house, Local District Trainer □ c) Off-site, Requiring In-state Travel 

□ b) In-house, Outside Consultant □ d) Off-site, Requiring Out-of-state Travel 

3 11; 



O 



Statements found as you continue Part 2 are intended to solicit your impressions of the different courses or activities 
you attended. You are to code your responses to the various viewpoints being suggested. This portion of the 
questionnaire asks for the level of your agreement or disagreement with each statement. 

Circle the number which best describes your response based upon the range provided: 

(SA) (A) (D) (SD) 

Strongly Agree Agree Disagree Strongly Disagree 

<D ® ® ® 



8. instructor and Presentation Materials Effectiveness SA JQ_ S D 

. ....■■■■v>».v.f 

A. Course instructor or facilitator (in the case of staff development / \ X f '% 
activities) was well prepared to conduct this training. / I \/ J 3\ 4 

B. Instructor/facilitator was knowledgeable in his/her understanding of the / /" 
subject matter presented. 1 2w \. : /3 4 

C. The handouts and audio/visual aids were very useful in helping me to 

understand the course. 1 \ \2" 3 4 

D. These materials will be useful as reference tools for implementing what I ... 

have learned and in sharing with my peers the training [.received. -t ' .' 2 3 4 



9. Implementation of Course Content 5 A A —EL- S Q 

A. I have used the skills/knowledge acquired tachahge my instructional, 

administrative, or service delivery rjiethod'a, ,a, k ^ 12 3 4 

B. I have used the skills/knowledge acqu^ intent in the 

classroom, revise administrati%.policies, or dhar^^pupll services I 12 3 4 

deliver. '\ v \*"--, "" v 

C. As a result of this course,,! have atter^tedt^i^ the attitudes and 

commitment of students I f each, or staff with whom I work. 12 3 4 



10. Summary of Course Outbom^s 5 A A_, , P 5 P 

A. Coufsd!cQfltent exceeded my expectations in terms of quality, 

relevance to d«^.lf foutihes, and attention to professional needs. 12 3 4 

B. Attendlfigltfis cou*£e/£ctivity increased my understanding of "nationally 

recognized'^ and administrative models. 12 3 4 

C. This course improved my ability to either teach specific content areas, 

perform administrative duties, and/or instruct other educators. 12 3 4 

D. Skills and knowledge l acquired have improved the motivation and 

performance of students I teach or staff I supervise. 12 3 4 

E. I feel confident that skills and knowledge I received will result in improved 

performance for students considered to be M at risk" of failure. 12 3 4 

F. As a result of my participation in this course, I now have a stronger 
commitment to teaching, administering, or providing pupil services in the 

public school system. 12 3 4 
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I WHEN YOU HAVE COMPLETED COURSE EVALUATIONS, PROCEED TO PART 3. 
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- Part 2 - 



INDIVIDUAL COURSE EVALUATION FORM 



Local Chapter 2 Coordinators have researched and identified all Chapter 2 training and staff development activities 
attended by their staff during the 1990-91 school year. You should possess sufficient copies of Part 2 in your packet 
to complete one assessment for each of a total of five (5) courses or training activities identified. 
Part 2 is a course-by-course overview of the professional and educational benefits you derived from various training or 
staff development activities. As respondents who understand assessment techniques, your perceptions are critical! 
All questions are to be answered, unless otherwise indicated. Thank you again for your helpl, :v, Xv 



Course Title g course #2 

1. 



What is the Course Title which best describes the training or staff development activity you attended? Use 
Appsnftix B: Listinn of Courses and Trainin g Components in completing this question. W ./ 
If you are unable to find a reasonable match to your Descriptive Course Title, please s^pply>an alternate. 

D TiJ'») eTi " e; ^"Ov'/ ' 

Alternate Title You Chose: , — 1 X. — — 



If you were required to identify arr alternate Course Title, please assist us in classifying its general content area 
from the listing below. Check (✓) only one box*. :: 

□ a) Instructional DeSvery □ e) Performance Assessment s, □ i ) Schoolwide Improvements 

□ b) Academic Education □ f) Staff Effectiveness trairtng □ j) School Environment 

□ c) Pupil Motivation □ g) I nstiuclonal Ceadf rship' □ k) Parental Involvement 

□ d) Behavioral Modification h) "Cwiicular ; ServiceV □ I ) Other : 




Course Purpose, Focus, Format, and v Aeieor|i^rirodatlons 

In answering questions 3 through 7, you may wish to : reV'to Apponriix A: Definition of Terms to assist you in selecting 
the correct response. Check (vO.only brie box per question. 

3 . Which of the following categories best describes the purpose of the training course you attended? 

□ a) Emptoyee Orientation vti c) Inservice Training □ e) Staff Development 

□ b). :i Pra8eTvic«Jraining.. w > □ d) Retraining 

4. Which of these categories best describes the focus of the training course you attended? 

□ a) Policiesf& Procedures □ d) Administrative Skills Training 

□ b) I nstrudjonaipf actices (If you have checked Instructional Practices, complete question No. 5 below.) 

□ c) NorHnstnJctional Methods □ e) Supportive Services Skills Training 

5. If the primary focus was Instructional Practices, check (✓) one box for each of the following inquiries: 

A. Designed to expand my knowledge of curriculum development? □ 1) Yes □ 2) No □ 3) Not sure 

B. Designed to assist me in the effective delivery of instruction? □ 1) Yes □ 2) No □ 3) Not sure 

C. Designed to offer state-of-the-art pupil assessment strategies? □ 1) Yes □ 2) No □ 3) Not sure 

6 . Which of these categories best describes the training format you experienced while attending this course? 

□ a) Structured Couree Work (Credit or Noncrodit) □ b) Conference □ c) Seminar □ d) Workshop 

7. Which of the these categories best describes the accommodations made for attendance at this course? 

□ a) In-house, Local District Trainer □ c) Off-site, Requiring In-state Travel 

□ b) In-house, Outside Consultant □ d) Off-srte, Requiring Out-of-state Travel 

3 " 120 



Statements found as you continue Part 2 are intended to solicit your impressions of the different courses or activities 
you attended. You are to code your responses to the various viewpoints being suggested. This portion of the 
questionnaire asks for the level of your agreement or disagreement with each statement. 

Circle the number which best describes your response based upon the range >rovided: 

(SA) (A) (D) <SD) 

Strongly Agree Agree Disagree Strongly Disagree 

® ® ® ® 



8. Instructor and Presentation Materials Effectiveness S A D -3-D. 



A. 


Course instructor or facilitator (in the case of staff dovelopment 
activities) was well prepared to conduct this training. 


.1 




5 3, 


4 


B. 


Instructor/facilitator was knowledgeable In his/her understanding of the 
subject matter presented. 


1 


2 -,„ 


3 


4 


C. 


The handouts and audio/visual aids were very useful in helping me to 
understand the course. 


1 


2 


3 


4 


D. 


These materials will be useful as reference tools for implementing what 1 
have learned and in sharing with my peers the training 1 received. 


1 


2 


3 


4 



Implementation of Course Content S A —A— _BL- S P 

A. I have used the skills/knowledge acauired to chary;? i^rjtfional, 

administrative, or service delivery methods, 1 2 3 4 

B. I have used the skills/knowledge acquired to teach new. content in the 

classroom, revise administraUvepoRcies, or change. P^P* ! services I 12 3 4 

deliver, 

C. As a result of this course,.! have attempted to : improve the attitudes and 

commitment of students I teach or staff with whom I work. 12 3 4 



1 o. Summary of Cours;e Outcomes S A ft -J2_ _3JL 

A. Course content exceeded niy expectations in terms of quality* 

relevance to daily routines, and attention to professional needs. 12 3 4 

B. Attending^ increased my understanding of "nationally 

recognized" education and administrative models. 12 3 4 

C. This course improved my ability to either teach specific content areas, 

perform administrative duties, and/or instruct other educators. 12 3 4 

D. Skills and knowledge I acquired have improved the motivation and 

performance of students I teach or staff I supervise. 12 3 4 

E. I feel confident that skills and knowledge I received will result in improved 

performance for students considered to be "at risk" of failure. 12 3 4 

F. As a result of my participation in this course, I now have a stronger 
commitment to teaching, administering, or providing pupil services in the 

public school system. 12 3 4 



i 




WHEN YOU HAVE COMPLETED COURSE EVALUATIONS, PROCEED TO PART 3. | 



- Part 3 - 



OVERALL ASSESSMENT OF TRAINING & 
PROFESSIONAL DEVELOPMENT 



Your responses to Part 3 provide ADE with an assessment, in summary, of your district's Program of Training & 
Professional Development, across all the courses you evaluated in Part 2. Please answer all questions! 



1. 



How many Chapter 2-supported courses have you attended during 1990-91 for which you are completing a 
separate Part 2 survey instrument today? 

Number of Part 2 surveys attached to this questionnaire: 



Mv primary reasons for participating in the district's Program of Training & PfcrfessioraLOovltopment include 
my professional need: Please check (✓) one box for'^ch o^tsjtteptftnts below. 

A. To acquire effective instructional delivery skills. □ 1>Y|s □ 2) Nob, j""^ 

B. To improve classroom management and effective d^esXi 2) fo p 3) |^w«ki^ 
recordkeeping skills. /. ''''""""'"--^ "\y 

C. To expand my knowledge of academic content- Q % Yes Q 2>*o □ 3) ' do n°t woric in an 
directly related to a current position. / \ j / %/ instructional capacity. 

D. To learn more about national/local mocJelkof '% 1)Yes □ 2) No □ 3) 

continuous pupil assessments. / { \ ■■■^\ ^ 

E. To learn awut innovations in the ^structiiring ot P^fYes □ 2) No □ 3) 

curriculum for effective teaehingand learning. '\ ^ < / instructional capacity. 

F. To provide more quality instruc&^r^ceC> □ 1)Yes □ 2) No □. 3) |*^S£Sclty. 
to at-risk pupils. "v.,. "" i % £ / 



The statements below are intln&To summa#your impressions of the district's training P^ a o ^ P' ease code 
your responses to the viewpoints suggested hf indicating the level of your agreement or disagreement. 

Circle the num^r* which best descries your response based upon the range provided: 

(S>AY^~^X ^ / (A) (D) ■ (SD) 

Strong^ Wrfee ' V A^gree Disagree Strongly Disagree 

.® 

S A A _D_ §D 




3. LEA's Program of training & Professional Development 

A. In general, the district's Chapter 2-supported Program of Training & Staff 
Development offers sufficient opportunities for employee orientation, staff 
training, and professional development. 

B. ... provides sufficient release time to avoid scheduling conflicts. 

C. ... offers opportunities to learn policies and procedures. 

D. ... offers opportunities to acquire effective instructional practices. 

E. ... provides training in non-instructional methods, such as enhanced 1 2 3 4 

classroom management strategies. 

F. % ... provides training in administrative skills enhancement. 

G. ... provides training in supportive services skills development. 

H. ... offers staff an adequate variety of training formats, such as workshops, 
seminars, conferences and structured course work. 

. 122 



2 3 4 

2 3 4 

2 3 4 

2 3 4 



2 3 4 
2 3 4 



APPENDIX A 

DEFINITION OF TERMS 



Definitions for Part 1, Question 5: 



Administrator 


Administrators include: 1) the superintendent and assistant superintendent; 2) professional staff 
who are directly supervised oy supermtenaenis or int?ir dbbibicuub, o; ymioiyciib ui viuc 
principals who also serve as district administrators; 4) nonclerical staff who^aictivities fall within 
the categories of Business and Fiscal Services (i.e., purchasing, payroll; personnel, inventory 
control, etc.) or Educational Services (i.e., test coordinators, evalu^tprs^plannei;^ special 
education directors, etc.); and 5) individuals, such as governing board meme^rg^h^arfi vested 
with responsibilities for educational planning and policy making. " V f \ 


Instructional 
Personnel 


Instructional Personnel include: 1) certified teachers whose primacy re^psi^fities are 
considered .to be delivery of instruction to pupils witntfi a variety oi classroom beuingb, 
2) individuals, such as teacher aides, who are employed or volunteer tftne Jo assist in the 
delivery of instruction to pupils; and 3) any other persons Who do nofi havevffiSponsibilities as 
administrators or support services staff and who spefKkthe majority *>f the school day within 
classrooms, helping teachers in their delivery of instruction and assisting! iaititramural activities. 


Support 

Services 

Staff 

(nonclerical) 

i 


Support Services Staff include: 1) school llbrartens.icounselors; and others assigned to 
consult with teachers and parents regarding learning problems, personal or social development, 
and educational/career choices; 2) staff .who providie psychofdgical services and work with 
educators in meeting the special , needs of students; 3) principals and other personnel 
responsible for attendance; 4) professionals who provide medical, dental, nursing and mental 
health services; and 5) pupil services staff who work to prevent or solve dudH problems at home, 
in the school, or within the community. 


Definition for Part 1, Question 13: *\ Vw ^ v 


At-Rlsk Pupil 


! At-Risk Pupife include individuate. who have dropped out of school as well as students who are 
currently enrolled, but have identifiable "at-risk H characteristics, including academic and economic 
factors that are-,rec'ognized as increasing the likelihood of their dropping out of the educational 
system. 


Definitions for Part 2, Question 3: 


Employee 
Orientation 


Employee orientation is a program of familiarization with the educational mission and 
organizational goals of the district. It includes: 1) the objectives being pursued by individual 
schools; 2) academic and vocational programs being offered students; 3) specialized programs 
being undertaken to solve problems or advance the educational climate of a school; 4) changes 
in policies and procedures; and, in the case of new employees, information critical to the 
professional success of the individual. 
Employee orientation is usually conducted once each year. 


Preservice 
Training 


Preservice training covers formal instruction which prepares an employee to assume new duties 
or in the case of educational personnel, receive certification or endorsement in particular subject 
areas or administration. Preservice train ng may incorporate career concepts associated with 
these new responsibilities. 



ERIC 



6 



Definitions for Part 2, Question 3 (Continued): 



Inservlce 
Training 


Inservice training covers instruction delivered to staff who are in need of new concepts, content, 
strategies, or other job-related knowledge in a current position or withir^a current area of 
responsibility. /' 


Retraining 


Retraining covers instruction delivered to educational staff in areas wh e re tf ^rii ng h;a$ already 
occurred but, due to advances in technology or shifting emphases in nitional ^jgr|rrirning, is 
now lacking as the educator attempts to apply content which had,be£n deiiyer^lii ar|earlier 
period. Retraining includes upgrade training; however, it should n^encdft^ of the 
"follow-up" activities associated with the delivery of the initial jnstruction. \ 


Staff 

Development 


Staff development is the measurable growth of an employee in both :g|pei^L ; knowledge and 
aptitudes that relate directly to performance of daily rotitin.es and assist jhat individual in relating 
to peers and supervisors. Staff development generally cfoes ncftv.address^he types of training 
listed above because acquisition of specific concepts, content, strategies, etc. is not its primary 
goal. 



Definitions for Part 2, Question 6: 



Conference 


A conference is a training fdrfn^t fehar^teri26d ; ;by a multi-disciplinary agenda that permits 
presentation and analysis of infoifoaftoa, as weft as formal interchange of views. In general, 
participants are offer^*^ seisibns by recognized leaders in their field; breakout 
sessions which permit th^indiyiau^l tqlchoose topics of particular interest; and panel 
discussions that,,encourage (^jslku]»^arti8 trainee participation. Conferences usually publish 
materials which o'uttiiie and elplai^all sessions while providing handouts and detailed 
information foaeaph sessidttiJtiat .^Spaifticipant attends. 


Seminar 


A seminar is a V^ta^9^ & OTat characterized by a group of individuals studying, usually in one 
major/topical area,. under t&e leadership of a professor or trainer with each doing research and all 
exchanging results, te one or more meetings, through reports and discussions. 


Structured 
Course Work 


f ' A : stmctured cdursfc is>either for credit or is taken nmcredit and is pursued at an institution of 
| higher educatiompriii affiliation with a university or college. 


Workshop 


A,worftshopJs i brief, intensive educational or training program for a relatively small group of 
individuals engaged in a specialized field or discipline which emphasizes participation in problem- 
solvihgefftirts designed to reach consensus as to the information and skills being acquired. 




APPENDIX B 

LISTING OF COURSES AND TRAINING COMPONENTS 



Descriptive Course Titles 


Course 
No. 


INSTRUCTIONAL DELIVERY: 

111 \J 111 %0 \S I ■ 1 « * \ m& !■ I V W ■ m ■ • 




Annlipd Content Enrichment 


101 


Effective/Innovative Instruction 


102 


Essential Elements of Instruction 


1 HQ 

1 Uo 


Learnina. Coooerative . . . 


104 


Learning, Mastery . . . 


105 


Instructional Stratpaips for At-Risk 




Stnripnfc 


106 


Lesson Planning 


107 


Research &Teachina Practices 


108 


Thpmatic Units 

i i iui i laiiu wi iuo 


109 


ACADEMIC EDUCATION: 




Basic Skills Through Music 


201 


English As A Second Language (ESL) 


202 


Great Minds - Great Works 


203 ? 


Health Education 


204 :: ' 


Higher uroer oKins (ininKingj 




Math ABCs of 


206 


Math Hands on 


20?: . 


Readina Across Curriculum 


. 208 4; : 


Social Studies Intearation X 

\J W 1 CA 1 v> k w V** 1 W vj MIL w W I wb k 1 W 1 L 


209 


Snprial Nppds in Lanouaae 




Whole Lanouaae 


211 :: 


Writing Across Curriculum 




PUPIL MOTIVATION: 




At-Risk Interventiort StrategiBS^ r ?\ 


301 


Assertive uis.cnpiine,- 




Building Up Kids 


304 


Counseling Strategies: ft)r Teachers 


305 


Human Effectiveness ,/ 


306 


Leadership - P^er Assisted 


307 


Leadership - Situational 


308 


Peer Support/Mentor Training 


309 


Problem Solving Techniques 


310 


BEHAVIORAL MODIFICATION: 




Chemical Abuse 


401 


PERFORMANCE ASSESSMENT: 




Assessment Methodologies/Strategies 


501 


Advanced Placement 


502 


Essential Skills Management 


503 



uebcnpiive v^ourse i muo 


Courso 
No. 


STAFF EFFECTIVENESS: 




Academic Content Survey (Asset) 


601 


Arizona's K-3 Acaderny ^%T^ 


602 


Career Management { \ \ f '\ 


603 


Classroom Management/ /> j % 


604 






& Materials) \, \y /' 


605 


InternatiQiili Reading Association 




Conference f\ i 


606 


Techfi:Qi<?.gy for.. Librarians arid Media 




Specialists 


607 


Time ManagfemenV 


608 


INSTRUCTIONAL LEADERSHIP & 




COACHING OF TEACHERS: 




ClinicaJ,,S,upervision 


701 


Coaching/Evaluation for 




Administrators 


702 


XjRstrtictional Leadership for 




/Secondary Principals 


703 


/S.T.A.R. (Strategies To Achieve 




Results) 


704 


CURRICULAR SERVICES: 




Curriculum Design & Implementation 


801 


Outcomes-based Education 




Outcomes-driven Developmental 




Model 


803 


SCHOOLWIDE IMPROVEMENTS: 




Effective Schools Research & 




Implementation 
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Descriptive Statistics 



Participant 

Study Group School Districts (N - 43) Responses 

Alhambra Elementary 30 

Amphitheater Unified 23 

Apache Junction Unified 30 

Catalina Foothills Unified 32 

Chandler Unified 26 

Chino Valley Unified 17 

Crane Elementary 12 

Creighton Elementary 24 

Deer Valley Unified 28 

Flagstaff Unified 15 

Flowing Weils Unified 32 

Ganado Unified 21 

Gilbert Unified 19 

Glendale Elementary 24 

Glendale Union High 30 

Indian Oasis-Baboquivari Unified 32 

Isaac Elementary 17 

Kyrene Elementary 34 

Lake Havasu Unified 15 

Laveen Elementary 18 

Litchfield Elementary I 7 

Madison Elementary 25 

Marana Unified 18 

Maricopa Unified 22 
Mesa Unified I 7 

Page Unified 24 
Paradise Valley Unified 36 
Pendergast Elementary 21 
Peoria Unified 20 
Phoenix Elementary 12 
Phoenix Union High 24 
Roosevelt Elementary 23 
Scottsdale Unified 26 
Sierra Vista Unified 40 
Sunnyside Unified 37 
Tempe Elementary 15 
Tempe Union High 20 
Tolleson Union High 18 
Tucson Unified *3 
Washington Elementary 23 
Williams Unified 15 
Yuma Elementary 9 
Yuma Union High * 9 

Unusable/No Response — 2. 

Total 975 
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Descriptive Statistics 
The Survey Sample 



Universe of Eligible Districts: 43 
Number of Participating Districts: 43 
Universe of Eligible Staff: 5,819 



Instructors 5,139 88.3% 

Administrators 334 5.7% 

Support Services 346 5.9% 



Number of Staff in Sample Set: 1,412 (24.2 percent of universe) 



Instructors 
Administrators 
Support Services 



N-Count 

1,212 
101 
99 



Percent of 
Universe 
Group 

23.5% 
30.2% 
28.6% 



Percent of 
Sample 
Group 

85.8% 
7.2% 
7.0% 



Response Rates 



District Responses: 43 100.0% 

Individual Responses: 975 68.9% 





Surveys 


Response 




Returned 


Rate 


Instructors 


804 


66.3% 


Administrators 


79 


78.2% 


Support Services 


_63_ 


62.6% 


945 


69.9% 


No Identifier 






Total Surveys 


975 


69.1% 



Individual Survey Return Rate: 



69.3% 
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Descriptive Statistics 



Part 1. Respondent Demographics (N = 975) 



Ql.l Please indicate your gender. 



Female 
Male 

Unusable/No Response 
Total 



Responses 

732 
222 

_21 

975 



Percent 

76.7% 
23.3% 



Q1.2 What is your race/ethnicity? 

Responses 

Asian/Pacific Islander 5 

Black 16 

Hispanic 53 

Native American/Alaskan Native 30 

White 841 

Unusable/No Response 30 

Total 975 



Percent 

0.5% 
1.7% 
5.6% 
3.2% 
89.0% 



Q1.3 Not applicable, pertains to respondent's work site. 



Q1.4 What is your highest level of educational attainment? 





Responses 


Percent 


High School Diploma 


24 


2.5% 


Associate Degree 


5 


0.5% 


Bachelor's Degree 


444 


46.8% 


Master's Degree 


434 


45.8% 


Doctorate 


24 


2.5% 


Post-Doctoral Study 


8 


0.8% 


Other 


9 


0.9% 


Unusable/No Response 


_2I 




Total 


975 
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Descriptive Statistics 



Part 1. Respondent Demographics (continued) 



Q1.5 What is your primary job classification at this time? 



Superintendent/Asst. Superintendent 

Principal/Asst. Principal 

Governing Board Member 

Other District Administrator 

Certified Teacher 

Teacher Aide 

Other Instructional Staff 

Librarian 

Counselor 

Other 

Unusable/No Response 
Total 



Responses 
12 
48 
1 
19 
774 
18 
12 
26 
17 
19 

-22. 
975 



Percent 

1.3% 
5.1% 
0.1% 
2.0% 
81.8% 
1.9% 
1.3% 
2.7% 
1.8% 
2.0% 



Q1.6 How long have you worked in your present position? 



Less than 1 year 
1 year, but less than 3 
3 years, but less than 5 
5 years, but less than 10 
10 years, but less than 15 
15 or more years 
Unusable/No Response 
Total 



Responses 

26 
196 
143 
182 
109 
158 

lfil 

975 



Pel cent 

3.2% 
24.1% 
17.6% 
22.4% 
13.4% 
19.4% 



Ql.7 Are you a member of a site-based school improvement team? 

Yes No 



Responses: 
Percent: 



299 
38.1% 



486 
61.9% 



Unusable/ Total 
No Response 

190 975 



Descriptive Statistics 

Parti. Respondent Demographics (continued) 

Qi ft Do vou narticiuate in the planning or implementation of schoolwide improvements (i.e the 
Q m^ess "cuTun? restructuring or other educational reforms) at either the district or local 



process i 
level? 



Yes No Unusable/ Total 

No Response 

Responses: 551 228 196 975 

Percent: 70.7% 29.3% 



Note: Questions 1.9 - 1.13 pertain only to classroom teachers or teacher aides. 

Q1.9 What is the average number of students per class to whom you provide instruction? 

Min. Max. Mean Std. Dev. Total Responses 

2.0 50.0 25.1 6.8 766 

Q1.10 Which grade level describes the majority of students with whom you interact daily? 

Responses Percent 

70 9.6% 



1 
2 
3 
4 
5 
6 
7 
8 
9 

10 
11 
12 

Preschool 
Kindergarten 

Unusable/No Response 349. 



78 10.7% 

78 10.7% 

79 10.9% 
59 8.1% 
45 6.2% 
53 7.3% 
42 5.8% 
49 6.7% 
64 8.8% 
44 6.1% 
16 2.2% 

5 0.7% 



Total 



44 6.1% 

A3. 
975 
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Descriptive Statistics 

Part 1. Respondent Demographics (continued) 

Ql.ll How many years total have you worked in instruction? 





Responses 


Percent 


Less than 1 year 


3 


0.4% 


1 year, but less than 3 


73 


9.3% 


3 years, but less than 5 


87 


11.1% 


5 yeare, but less than 10 


165 


21.0% 


10 years, but less than 15 


159 


20.2% 


15 or more years 


299 


38.0% 


Unusable/No Response 


122. 




Total 


975 





Q1.12 In which primary academic/vocational area do you deliver or assist with instruction? 





Responses 


Percent 


Preschool - Grade 6 (all subjects) 


351 


45.2% 


Math and Science 


97 


12.5% 


Language Arts 


110 


14.2% 


Vocational Education 


36 


4.6% 


Foreign Languages 


9 


1.2% 


Humanities 


20 


2.6% 


Performing and Creative Arts 


31 


4.0% 


Other 


123 


15.8% 


Unusable/No Response 


198 




Total 


975 





Q1.13 Are a majority (50% or more) of students in any of your classrooms considered to be "at risi 

Yes No Unknown Unusable/ Total 

No Response 

Responses: 263 412 100 200 975 

Percent: 33.9% 53.2% 12.9% 
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Descriptive Statistics 



Part 2. Individual Course Evaluations 



02 1 What is the Course Title whidi best describes the training or staff development activity you 
attended? Use Appendix B. Listing of Courses and Training Components in completing this 
question. 

If you are unable to find a reasonable match to your Descriptive Course Title, please supply an 
alternate. 



Descriptive Course Title 
Instructional Delivery 

Applied Content Enrichment 
Effective/Innovative Instruction 
Essential Elements of Instruction 
Learning, Cooperative... 
Learning, Mastery... 

Instructional Strategies for At-Risk Students 

Lesson Planning 

Research & Teaching Practices 

Thematic Units 
Academic Education 

Basic Skills Through Music 

English as a Second Language (ESL) 

Great Minds— Great Works 

Health Education 

Higher Order Skills (Thinking) 

Math, ABCs of... 

Math, Hands on- 
Reading Across Curriculum 

Social Studies Integration 

Special Needs in Language 

Whole Language 

Writing Across Curriculum 
Pupil Motivation 

At-Risk Intervention Strategies 

Assertive Discipline 

Building Up Kids 

Counseling Strategies for Teachers 

Human Effectiveness 

Leadership— Situational 

Peer Support/Mentor Training 

Problem-Solving Techniques 
Behavioral Modification 
Chemical Abuse 



Total 
Responses 



Cumulative 
Percent 



21 


2.1% 


38 


3.7% 


121 


11.9% 


101 


9.9% 


32 


3.1% 


12 


1.2% 


4 


0.4% 


11 


1.1% 


20 


2.0% 


2 


0.2% 


61 


O.0% 


10 


1.0% 


2 


0.2% 


31 


3.0% 


1 


0.1% 


30 


2.9% 


11 


1.1% 


7 


0.7% 


6 


0.6% 


24 


2.4% 


60 


5.9% 


25 


2.4% 


8 


0.8% 


29 


2.8% 


8 


0.8% 


24 


2.4% 


4 


0.4% 


4 


0.4% 


o 


0.3% 


22 


2.2% 
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Descriptive Statistics 



Part 2. Individual Course Evaluations (continued) 



Q2.1 What is the Course Title which best describes the training or staff development activity 
attended? (continued) 

Performance Assessment 

Assessment Methodologies/Strategies 

Essential Skills Management 
Staff Effectiveness 

Academic Content Survey (Asset) 

Arizona's K-3 Academy 

Career Management 

Classroom Management 

Instructional Technology (Computers & Materials) 

International Reading Association Conference 

Technology for librarians and Media Specialists 

Time Management 
Instructional Leadership & Coaching of Teachers 

Clinical Superviaion 

Coaching/Evaluation for Administrators 

Instructional Leadership for Secondary Principals 
Curricular Services 

Curriculum Design & Implementation 

Outcomes-based Education 
Schoolwide Improvements 

Effective Schools Research & Implementation 

Strategic Planning for Improvement 

Schoolwide Improvement Process 
School Environment 

Techniques in Safe & Orderly Environment 

Unusable/No Response 
Total 



31 


3.0% 


8 


0.8% 


1 


0.1% 


4 


0.4% 


10 


1.0% 


30 


2.9% 


32 


3.1% 


-9 

1 


A 1 Of 
0.1% 


2 


0.2% 


3 


0.3% 


19 


1.9% 


3 


0.3% 


2 


0.2% 


41 


4.0% 


4 


0.4% 


35 


3.4% 


18 


1.8% 


40 


3.9% 


4 


0.4% 


691 




1,711 
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Descriptive Statistics 

Part 2. Individual Course Evaluations (continued) 

Q2.2 If you were required to identify an alternate Course Title, please assist us in classifying its 
general content area from the listing below. 





Responses 


Percent 


Instructional Delivery 


452 


30.1% 


Academic Education 


255 


17.0% 


Pupil Motivation 


146 


9.7% 


Behavioral Modification 


70 


4.7% 


Performance Assessment 


72 


4.8% 


Staff Effectiveness Training 


178 


11.8% 


Instructional Leadership 


37 


2.5% 


Curricular Services 


66 


4.4% 


Schoolwide Improvements 


108 


7.2% 


School Environment 


28 


1.9% 


Parental Involvement 


4 


0.3% 


Other 


88 


5.9% 


Unusable/No Respou&e 


207 




Total 


1,711 





Q2.3 Which of the following categories best describes the purpose of the training course you 
attended? 





Responses 


Percent 


Employee Orientation 


87 


5.1% 


Pre service Training 


84 


4.9% 


In-Service Training 


823 


48.1% 


Retraining 


88 


5.1% 


Staff Development 


629 


36.8% 


Total 


1,711 





Q2.4 Which of these categories best describes the focus of the training course you attended? 





Responses 


Percent 


Policies & Procedures 


75 


4.5% 


Instructional Practices 


1161 


69.4% 


Non- Instructional Methods 


217 


13.0% 


Administrative Skills Training 


70 


4.2% 


Supportive Services Skills Training 


151 


9.0% 


Unusable/No Response 


37, 




Total 


1,711 
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Descriptive Statistics 



Part 2, Individual Course Evaluations (continued) 



Q2.5 If the primary focus was Instructional Practices, check yes or no for each of the follow 
inquiries: 

A- Designed to expand my knowledge of curriculum development? 

Yes No Not Sure Unusable/ Total 

No Response 

Respor ses: 688 270 62 691 1,711 

Percent: 67.5% 26.5% 6.1% 



B. Designed to assist me in the effective delivery of instruction? 

Yes No Not Sure Unusable/ Total 

No Response 

Responses: 1045 41 22 603 1,711 
Percent: 94.3% 3.7% 2.0% 



C. Designed to offer state-of-the-art pupil assessment strategies? 

Yes No Nat Sure Unusable/ Total 

No Response 

Responses: 493 397 123 698 1,711 
Percent: 48-7% 39.2% 12.1% 



Q2.6 Which of these categories best describes the training format you experienced while atten< 
this course? 

Responses Percent 

Structured Course Work 

(Credit or Noncredit) 278 16.5% 

Conference 145 8.6% 

Seminar 304 18.1% 

Workshop 955 56.8% 
Unusable/No Response 29 

Total 1,711 
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Descriptive Statistics 

Part 2. Individual Course Evaluations (continued) 

Q2.7 Which of these categories best describes the accommodations made for attendance at this 
course? 

Responses Percent 

In-house, Local District Trainer 795 46.5% 

In-house, Outside Consultant 579 33.8% 

Off-site, Requiring In-State Travel 321 18.8% 

Off-site, Requiring Out-of-State Travel 1£ 0.9% 

Total 1*711 

Q?.8 Instructor and Presentation Materials Effectiveness 

A. Course instructor or facilitator (in the case of staff development activities) was well 
prepared to conduct this training. 

Strongly Agree Disagree Strontrly Unusable/ Total 

Agree Disagree No Response 

1152 507 35 8 9 1,711 

67.7% 29.8% 2.1% 0.5% 

B. Instructor/faciHtator was knowledgeable in his/her understanding of the subject matter 
presented. 

Strongly Agree Disagree Strongly Unusable/ Total 
Agree Disagree No Response 

1223 464 15 2 7 1,711 

71.8% 27.2% 0.9% 0.1% 

C. The handouts and audio/visual aids were very useful in helping me to understand the 
course. 

Strongly Agree Disagree Strongly Unusable/ Total 
Agree Disagree No Response 

921 657 81 23 29 1,711 

54.8% 39.1% 4.8% 1.4% 
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Descriptive Statistics 



Part 2, Individual Course Evaluations (continued) 



Q2.8 Instructor and Presentation Materials Effectiveness (continued) 

D. The materials will be useful as reference tools for implementing what I have learned an< 
sharing with my peers the training I received. 

Strongly Agree Disagree Strongly Unusable/ Total 

Agree Disagree No Response 

828 666 148 39 30 1,711 

49.3% 39.6% 8.8% 2.3% 



Q2.9 Implementation of Course Content 

A. I have used the skills/knowledge acquired to change my instructional, administrative 
service delivery methods. 

Strongly Agree Disagree Strongly Unusable/ Total 

Agree Disagree No Response 

619 832 191 52 17 1,711 

36.5% 49.1% 11.3% 3.1% 



B. I have used the skills/knowledge acquired to teach new content in the classroom, re- 1 
administrative policies, or change pupil services I deliver. 

Strongly Agree Disagree Strongly Unusable/ Total 

Agree Disagree No Response 

565 804 265 56 21 1,711 

33.4% 47.6% 15.7% 3.3% 



5° 



. As a result of this course, I have attempted to improve the attitudes and commitmen 
students I teach or staff with whom I work 

Strongly Agree Disagree Strongly Unusable/ Total 

Agree Disagree No Response 

696 790 162 42 21 1,711 

41.2% 46.7% 9.6% « 2.5% 
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Descriptive Statistics 

Part 2. Individual Course Evaluations (continued) 

Q2.10 Summary of Course Outcomes 

A. Course content exceeded my expectations in terms of quality, relevance to daily routines, 
and attention to professional needs. 

Strongly Agree Disagree Strongly Unusable/ Total 

Agree Disagree No Response 

493 818 339 47 14 1,711 

29.1% 48.2% 20.0% 2.8% 

B. Attending this course/activity increased my understanding of "nationally recognized" 
education and administrative models. 

Strongly Agree Disagree Strongly Unusable/ Total 

Agree Disagree No Response 

463 825 342 59 22 1,711 

27.4% 48.8% 20.2% 3.5% 

C. This course improved my ability to either teach specific content areas, perform administra- 
tive duties, and/or instruct other educators. 

Strongly Agree Disagree Strongly Unusable/ Total 

Agree Disagree No Response 

529 804 312 49 17 1,711 

31.2% 47.5% 18.4% 2.9% 

D. Skills and knowledge I acquired have improved the motivation and performance of students 
I teach or staff I supervise. 

Strongly Agree Disagree Strongly Unusable/ Total 

Agree Disagree No Response 

532 834 279 46 20 1,711 

31.5% 49.3% 16.5% 2.7% 

E. I feel confident that skills and knowledge I received will result in improved performance for 
students considered to be "at risk" of failure. 

Strongly Agree Disagree Strongly Unusable/ Total 
Agree Disagree No Response 



556 807 275 49 24 1,711 

33.0% 47.8% 16.3% 2.9% 
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Descriptive Statistics 



Part 2. Individual Course Evaluations (continued) 



Q2.10 Summary of Course Outcomes (continued) 

F. As a result of my participation in this course, I now have a stronger commitmer 

teaching, administering, or providing pupil services in the public school system. 

Strongly Agree Disagree Strongly Unusable/ Total 

Agree Disagree No Response 

546 798 288 60 19 1,711 
32.3% 47.2% 17.0% 3.5% 
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Descriptive Statistics 

Part 3. Overall Assessment of Training and Professional Development 

Q3.1 How many Chapter 2-supported courses have you attended during 1990-91 for which you are 
completing a separate Part 2 survey instrument today? 

Number of Course Evaluation forms attached to this questionnaire? 

Min. Max. Mean Std. Dev. Total Responses 

0.0 5.0 1.9 1.3 951 

Q3.2 My primary reasons for participating in district's Program of Training & Professional 
Development include my professional need: 

A. To acquire effective instructional delivery skills. 

Yes No Do Not Unusable/ Total 

Instruct No Response 

Responses: 551 63 154 207 975 
Percent: 71.7% 8.2% 20.1% 

B. To improve classroom management and effective recordkeeping skills. 

Yes No Do Not Unusable/ Total 

Instruct No Response 

Responses: 421 183 154 217 975 
Percent: 55.5% 24.1% 20.3% 

C. To expand my knowledge of academic content directly to a current position. 

Yes No Do Not Unusable/ Total 

Instruct No Response 

Responses: 420 184 155 216 975 
Percent: 55.3% 24.2% 20.4% 

D. To learn more about national/local models of continuous pupil assessments. 

Yes No Do Not Unusable/ Total 

Instruct No Response 

Responses: 364 234 154 223 975 
Percent: 48.4% 31.1% 20.5% 
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Descriptive Statistics 



Part 3. Overall Assessment of Training and Professional Development (continue 



Q3.2 My primary reasons for participating in district's Program of Training & Professional 
Development (continued) 

E. To learn about innovations in the restructuring of curriculum for effective 
teaching and learning. 



Responses: 
Percent: 



Yes No Do Not Unusable/ Total 

Instruct No Response 

510 97 154 214 975 
67.0% 12.7% 20.2% 



F. To provide more quality instruction and/or services to at-risk pupils 

Yes No 



Responses: 
Percent: 



523 
68.6% 



85 
11.2% 



Do Not 
Instruct 

154 
20.2% 



Unusable/ Total 
No Response 

213 975 



Q3.3A. In general, the district's Chapter 2-supported Program of Training & Staff Development o: 
sufficient opportunities for employee orientation, staff training, and professional develops 



Strongly 
Agree 

313 
45.4% 



Agree Disagree 



322 
46.7% 



48 
7.0% 



Strongly 
Disagree 

7 

1.0% 



Unusable/ Total 
No Response 

285 975 



B. ...provides sufficient release time to avoid scheduling conflicts. 

Agree Disagree 



Strongly Agree Disagree Strongly 

Agree Disagree 

230 361 109 24 

31.8% 49.9% 15.1% 3.3% 



Unusable/ Total 
No Response 

251 975 



...offers opportunities to learn policies and procedures. 

Strongly Agree Disagree Strongly 

Agree Disagree 

224 379 101 7 

31.5% 53.3% 14.2% 1.0% 



Unusable/ Total 
No Response 

264 975 



ERIC 



24 



HI 



Descriptive Statistics 



Part 3. Overall Assessment of Training and Professional Development (continued) 



Q3.3 In general, the district's Chapter 2-supported Program of Training & Staff Development 
(continued) 

D. ...offers opportunities to acquire effective instructional practices. 

Strongly Agree Disagree Strongly Unusable/ Total 

Agree Disagree No Response 

334 358 33 5 245 975 

45.8% 49.0% 0.45% 0.7% 



E. ...provides training in non-instructional methods, such as enhanced classroom 
management strategies. 



Strongly Agree 
Agree 

236 375 

32.9% 52.2% 



Disagree 

.97 
13.5% 



Strongly 
Disagree 
10 
1.4% 



Unusable/ Total 
No Response 

257 975 



.provides training in ad^ninistrative skills enhancement. 

Strongly Agree Disagree Strongly 

Agree Disagree 

142 318 184 27 

21.2% 47.4% 27.4% 4.0% 



Unusable/ Total 
No Response 

304 975 



G. ...provides training in supportive services skills development. 

Agree Disagree 



Strongly Agree Disagree Strongly 

Agree Disagree 

162 376 129 14 

23.8% 55.2% 18.9% 2.1% 



Unusable/ Total 
No Response 

294 975 



H. ...offers staff an adequate variety of training formats, such as workshops, seminars, 
conferences and structured course work, 



Strongly 
Agree 

268 
37.2% 



Agree Disagree 



324 
44.9% 



115 
16.0% 



Strongly Unusable/ Total 

Disagree No Response 

14 254 975 
1.9% 
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Summary op Statistical Findings 

Part 1. Respondent Demographics 

A. All Respondents 

• 76 percent female 

• 89 percent whit/j 

• 93 percent hold a Bachelor's or Master's Degree 

• Years Employed in Current Position: 

45 percent 5 years or less 
55 percent more thsm 5 years 

• Involvement in Site-Based/Schoolwide Improvement Activities: 

62 percent were not members of site-based improvement teams 
71 percent do participate in schoolwide improvement aclivitic* 

B. Teacher Respondents Only 

• Average Class Size: 25 

• Grades Taught: 

63 percent teach Preschool (PS) - 6th grade 
37 percent teach 7 - 12th grade 

• Teaching Experience: 

58 percent have been teaching longer than 10 years 
21 percent have been teaching less than 5 years 

• Subject Areas: 

45 percent teach general subjects, grades PS - 6th grade 

13 percent teach math/science 

14 percent language arts 

• At Risk Students: 

One third of the respondents stated that at least 50 percent of their students were 
considered to be at risk of academic failure. 

O 
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Summary of Statistical Findings 

Part 2. Overall Assessment of Chapter 2 Program Activities 

(All Respondents) 

A. Average Number of Chapter 2-funded Courses Attended: 1.9 

B. Primary Reasons for Participating in District Training and Professional Development 
(T&PD) Activities: 

1. Highest Positive Ratings: 

72% -To acquire effective instructional delivery skills 
fi9% -To provide more quality instruction and /or services to at-risk pupus 
m -To E about Novations in the restructuring of curriculum for effective teaching 
and learning 

2. Highest Negative Ratings: 

31% -Do not participate in T&PD activities to learn about national/local models of 
continuous pupil assessments. n ce nn ^„ a 

24% -Do not participate in T&PD to improve classroom management and effective 
record-keeping skills , . , . , 

24% -Do not participate in T&PD to expand their knowledge of academic content 
directly related to current position. 

C. Impressions of the LEA Program of Training and Professional Development 

Overall 92% of the respondents stated that they either "Strongly Agreed" (45.4%) or "Agreed'' 
(46 7%) Sat local Chapter 2-supported programs of T&PD offered sufficient opportunities for 
employee orientation, staff training and professional development. 

1. Highest "Strongly Agree" 

46% opportunities to acquire effective instructional practices. 

2. Highest "Strongly Disagree* 

4% provides training in administrative skills enhancement. 

The lowest combined negative (Disagree ;,A Strongly Disagree) ratings were reported for: 

31% provides traimng in administrative skills. 

18% provides sufficient release time to avoid scheduling conflicts. 
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Summary of Statistical Findings 



Parts. Individual Course Evaluations 



A* Course Characteristics 

• Number of courses reported: 1,568 

• Variety of Courses Attended 

Variations in the type of training courses attended under Chapter 2-funded T&PD 
programs were considerable. Fifty-four separate titles which respondents could choose in 
identifying the courses they attended were listed in the survey appendix. Of these, four 
course titles accounted for 30 percent of trainee responses: 

1 2% -Essential Elements of Instruction 

10% -Cooperative Learning 

6% -Writing Across C rriculum 

6% -English as a Sect ud Language 

However, selections from the existing list of course titles amounted to only 65 percent of ail 
responses. Thus, 35 percent of the respondents reported attending additional T&PD courses/ 
activities not initially identified by the Arizona Department of Education. 

• Global Content Classification of Courses Attended 

30 percent of all courses attended (regardless of course title) were categorized under the 
global content heading of Instructional Delivery. The top four areas reported were: 

30% -Instructional Delivery 

17% -Academic Education 

12% -Staff Effectiveness Training 

10% -Pupil Motivation 

• Purpose of T&PD Activity— In-service Training vs. Staff Development 

48 percent of the respondents identified the activity as being In-service Training (concerned 
with acquisition of new concepts and/or new content areas). 

37 percent identified the activities attended as StaffDeveloprnent (concerned with improving 
general knowledge and aptitudes that relate to performance of daily routines) 

• Focus of T&PD Activity 

70 percent of all respondents identified Instructional Practices as the primary focus of the 
T&PD activity attended. 

94 percent of these identified Delivery of Instruction as the primary design of the course. 
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Summary of Statistical Findings 



• Format of T&PD Activity 

56 percent of all T&PD activities attended were considered to be Workshops, as opposed 
Seminars (18%), structured course work (17%) or conferences (9%). 

• Location of T&PD Activity 

80 percent of the courses were held In-house. 



B. Course Evaluations 

1. Instructor and Materials 

a) Preparation and Knowledge 

Domain: The instructor was well prepared to conduct the T&PD session and was 



Result: 



knowledgeable of the subject matter. 

Respondents gave an overwhelmingly positive (98%) rating to the preparation 
and knowledge of the instructors conducting T&PD sessions. 



Positive 
Negative 



Strongly 
Agree/Disagree Agree/Disagree 

28.5% 69.8% 



1.4% 



.3% 



Combined 

98.3% 
1.7% 



b) Handout Materials 

Domain: Tne handout materials were useful as learning tools and as reference materials 
to share with peers. 

Respondents were slightly less positive about materials used in various T&PD 
activities, as represented by lower "Strongly Agree" responses and an overall 



Result: 



higher combined negative score 
Agree/Disagree 



Positive 
Negative 



39.4 
6.8 



Strongly 
Agree/Disagree 

52.1 
1.7 



Combined 

91.5 
8.5 



2. Implementation of Course Content 
a t Changes in Activities 

Domain: Attendance at T&PD activities resulted in changing j^tnnroved 
administrative methods, the content or policies previously used or an unproved 
attitude/commitment of students or staff. 
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Summary of Statistical Findings 



Result: Nearly 15% of the respondents did not agree that positive changes occurred as 
a result of attending various T&PD activities. 

Strongly 

Agree/Disagree Agree/Disagree Combined 

Positive 47.8% 37.0% 84.8% 

Negative 12.2% 3.0% 15.2% 

3. Summary of Course Outcomes 

a) Effects on Instruction and Student Outcomes 

Domain: Outcomes of attending the T&PD activities included improved abilities to teach 
specific content areas, improved motivation and performance of students and 
positive impact on the performance of at risk students. 

Result: While responses on the impact T&PD activities had on teaching ability and on 
students was very positive, respondents indicated that this was not true for 20 
percent of the courses taken. Based on the distribution of negative responses in 
this survey, 20 percent reflects a significant negative result. 

Strongly 

Agree/Disagree Agree/Disagree Combined 
Positive 48.2 31.9 80.1 
Negative 17.1 2.8 19.9 



b) Commitment and Understanding 

Domain: Outcome of attending T&PD activity was an increase in commitment to job and 
an increased understanding of education and administrative models. 

Result: 78 percent of the respondents agreed with the impact T&PD activities had on 
their understanding of education and administrative models and an increased 
commitment to their job. However, over 22 percent disagreed with these 
propositions. 

Strongly 

Agree/Disagree Agree/Disagree Combined 
Positive 48.0 29.9 77.9 

Negative 18.6 3.5 22.1 



c) Relevance 

Domain: The course content exceeded my expectations in terms of quality, relevance to 
daily routines and attention to professional needs. 



Summary of Statistical Findings 

Result: 77 respondents agreed with the statement on relevance of T&PD activities to 
daily routines and professional needs. However, 23 percent replied that they did 
not agree with the statement. 

Strongly 

Agree/Disagree Agree/Disagree Combined 
Positive 48.2 29.1 77.3 
Negative 20.0 2.8 22.8 
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Summary of Statistical Findings 



Disaggregated Data 



Instructional personnel self-report of having 50% or more at-risk pupils in their classes. 



Studv OrouD School Districts 


Yes 


No 


Unknown 


Allmmhffi RlpmpntiiTV 


10 


43.5% 


8 


34.8% 


5 


21.7% 
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6 


31.6% 


11 
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2 


10.5% 


An ache Junction Unified 


3 


11.5% 


15 


57.7% 


8 


30.8% 


\jtXkMXiiiitx f UUU11119 u luiicu 


3 


13.6% 


18 
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1 


4.5% 
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XI 


42.3% 


13 
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2 


7.7% 
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3 
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8 
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5 


31.3% 
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3 
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5 


45.5% 


3 
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13 
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1 
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13 


65.0% 


4 


20.0% 




5 


45.5% 


6 
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0 
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18 
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11.1% 
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.0% 


Gilhprt Unified 

\iilUCt b U11111CLI 


2 


14.3% 


11 


78.6% 


1 


7.1% 


OlpnHnlp T<llpTnpni'JiT F v 

VJ lClldCllC U1CU1C11 LAI Jr 


7 


35.0% 


19 


fin n% 


1 


5.0% 


CrlpnflAlp Union Hi^h 

VJICllUalC (J 111U11 lAlgll 


8 


33.3% 


13 


54.2% 


3 


12.5% 


TnHinn Ofl<?i<i-RfinofinivATn Unifipd 
xiimciii v/floii} iiauuuuivuii uiiuicu 


18 


75.0% 


1 


4.2% 


5 


20.8% 


Tqaap Rlpmpnfcarv 

IDuuL U1CU1CU uul Jr 


10 


66.7% 


3 


20.0% 


2 


13.3% 


TCvTPnP TTIptti pn f_Q TV 


3 


11.5% 


22 


84.6% 


1 
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Madison Elementary 
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5.9% 
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.0% 


Phoenix Union High 
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Roosevelt Elementary 
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6 
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Scottsdale Unified 
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38.5% 
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Sierra Vista Unified 


6 
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Sunnyside Unified 


15 
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35.7% 


3 
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Tempe Elementary 


7 


50.0% 


4 
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3 
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Tempe Union High 


5 
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6 
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1 


8.3% 


Tolleson Union High 


4 
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9 
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2 
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Tucson Unified 
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1 


11.1% 


Washington Elementary 


7 
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1 


4.5% 


Williams Unified 


2 
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6 


66.7% 


1 
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Yuma Elementary 


1 


12.5% 


7 


87.5% 


0 


.0% 


Yuma Union High 


_3. 


26.3% 


JJL 


57.9% 


_Ji 


15.8% 


Total 


263 




412 




100 
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ARIZONA DEPARTMENT OF EDUCATION 

Research & Development Division 
1535 W. Jefferson 
Phoenix, Arizona 85007 



CHAPTER 2 COORDI NATOR QUESTIONNAIRE 

- District Programs of Training & Professional Development - 



Purpose: This questionnaire Is part of an overall evaluation of training apd professiona 
development activities funded through Chapter 2 during the PQ^l school 
year. Your responses will help the Arizona Department of Education to 
evaluate effectiveness of program activities administered dHhe distttef If vej 

Organization: This questionnaire is divided into seven pWts. Part H requests 
demographic information about the respondent,... Part 2 examines district 
mechanisms for authorizing Chapter 2.,, training and professional 
development activities. Part 3 investigates ' the district's, -process for 
approving staff training. Part 4 examines tocatty designed evaluation 
and strategic planning processes. Pdrr S asks for your impressions of the 
state's Chapter 2 Program. Part 6 requests feedback .regarding LEA/SEA 
interactions. Finally, Part 7 asks forsug^estiohs to improve the delivery of 
services by the State Chapter- ^Office. \ : 

Do aai complete this questionnaire: un/e^ y'ou are a local Chapter 2 
Coordinator or have format rosponsfbtflHes for organizing delivery of 
training and professional d^yelopifnent ^ctlvltles to district personnel. 

Please begin by a^wwlhg^questlons which help distinguish you in 
terms of .Job f unctlonX You ?are ; encouraged to respond In an open 
manner.'' ftble, that all ihtormrflon obtained from this questionnaire will 
be kept cpnflo^rrttal and; reported only In state-aggregated form. 

Thank you for your assistance. 



Today's 1 date,:' 



District of employment;:: 



- Part 1 - 
Respondent Demographics 



1 . Please indicate the number of years you have been responsible for coordinating training 
and professional development activities for your district. — Y r s- 

2. is coordination of Chapter 2 projects and activities your primary job? □ a) Yes □ b) No 

3 . Have you ever been a classroom teacher? □ °) Yes a b) No 
■f If yes, how many total years of experience do you have as a classroom teacher? — yrs. 

, is: 



- Part 2 - 

Focus and Training Committee Review of Activities 



By checking (✓) all the items that apply to your Chapter 2-supported program of Training 
Professional Development, please indicate whether your district uses a committee 
structure for purposes of: 



□ a) Determining general policy and 

procedures. 

□ b) Formulating long-range strategic 

plans. 

□ c) Reviewing "applications" or requests 

by staff for training and professional 
development activities. * 



□ d) Determining which types of courses 

will be made available to staff 

□ e) Setting limits on the amount of funds 

to be utilized fOF§'aefe|. trainee. 

□ 0 A structured ebm^lfFde is^not used 

for any of these p|jfippsesV : <lf you 
select this response, <iMesJion 2 
below.) "\ \ ' J / 



If your district uses a committee structure to plan and/or approve expenditures for training 
and professional development, indicate by checking (✓rail that apply^rnembership 
represented on that committee; 



ADMINISTRATORS 

□ a) Superintendent/ Assistant 

Superintendent 

□ b) School Principal/ Assistant 

Principal 

□ c) Governing Board Member 

□ d) Other District Administrators 



INSTRUCTIONAL PERSONNEL 

□ e) Certified Teacher . 

□■■ f); Teacher Aide, 

£l g) Other Instructional Staff 



OTHERS 

□ h) Parents 

□ i) Private School 

Representatives 

□ j) Members of Sch 

Business Partners 



How many individuals serve on,thi$ : district Chapter 2 review committee? 



GO 



Consider the different methods by,whlch Chapter 2 training activities are made available 
to personnel in your district. Expressed as a proportion of the total Training & Professional 
Development Program,/ indicate the amount of emphasis your district places on the 
following preas (answers should total 100%): 

a) Mirii-grantsI for. innovative staff-initiated training. 

b) dist?|ct^dndated. : inservice (either on-site or off-site). 

c) Staff-requested inservices, NOT sponsored or 
mdndated by the District. 

d) OtherC "^ : '' ■ 

e) Other: — 

100% 



% 
% 

% 
% 
% 



4. What would you say is the primary focus of Chapter 2-funded Iraining and staff 
development activities in your district? Check (✓) only one: 



□ a) Improved Instructional 

Delivery 

□ b) Enhanced Educational 

Content 

□ c) More Effective Student 

Motivation 

□ d) Improved Assessments 



□ e) Increased Staff 

Effectiveness 

□ f) Enhanced Instructional 

Leadership 

□ g) Improved Teacher 

Coaching 

□ h) Improved Curriculum 

2 X 5^ 



□ i) Schoolwide 

Improvements 

□ j) Better School 

Environment 

□ k) Increased Parentc 

Involvement 



0 
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- Part 3 - 
District Process for Approving Staff Training 

1 . Are district personnel required to submit a formal application for Chapter 2-supported 
training and professional development activities? 

□ a) Yes □ b) No (If V«. pl«aw attach a copy of th« training application form.) 

2. Is there a formal review process which must be addressed before training applications 
can be approved? 

□ a) Yes □ b) No 

3 . Who makes the final decision to fund an Individual's request for qtfei^aHc^ at|ttaining? 

' □ a) District Superintendent □ c) Staff Developer □ e),Qtherf ^ 2 J j. 

□ b) Chapter 2 Coordinator □ d) Committee Members \ \/S 



4. 



How many applications did you receive during the l^O^l schoo! yeanfof participation in 
Chapter 2-funded staff training activities? 

Number of applications (not number of persons): — Ca) 

Of applications received, how many were NOT approved for funding: <m 

(If you do not have access to a specific number, you may estimate the 
percentage not approved during the 19<?€Mf1 school year.) ^ 

Has your district established a formal limit oh- the dolldr amount any one individual may 
expend in attendance at Chapter 2-funded staff training activit.es? □ a) Yes □ b) No 
If yes, what is this amount?C$^~- p^ ^IvfaW and/or $ per course or activity. 



(c) : 



CcO 



6. 



Are there any district personnel w^o aretfequired to participate in specific training and staff 
development activities (for, example,. new employee orientation)? 
□ a) Yes □ b) No t^o, skip the following section.) 

If Yes, indicate by checking M all that apply, categories which best describe > those 
individuals required to, participate in some type of Chapter 2-funded training and 
professional during the 1990-91 school year: 



\ Administrators 

□ a) New School Principals/ 

Assistant... Principals 

□ b) Currert-School Principals/ 

Assistant Principals 

□ c) New Governing Board 

Members 

□ d) Current Governing Board 

Members 

□ e) Other New or Current 

District Administrators 



INSTRUCTIONAL PERSONNEL 

□ f) New Certified Teachers 

□ g) Current Certified Teachers 

□ h) New Teacher Aides 
i) Current Teacher Aides 



j) other New or Cunent 
Instructional Staff 



NONCLERICAL SUPPORT 

□ k) New Librarians 

□ I) Current Librarians 

□ m) New Counselors 

□ n) Current Counselors 

□ o) Others 

(Including psychological 
attendance, health, and 
other pupil services staff. 



7. Are there any subject areas that require instructional personnel to receive training in new 
content or innovative instructional methods on an annual basis? □ a) Yes u Q) inc 



If yes, name three subject areas in order of training priority: 
1) 2 > 



3) 



- Part 4 - 

District Training Evaluation and Planning Processes 
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1 . Does your district evaluate, in a formal and somewhat technical manner, the training and 
professional development activities attended by district personnel? □ a) Yes □ b) N< 

2 . Considering your district's conduct of evaluations, select by checking (✓) the primary 
method by which staff training evaluations are obtained: 

□ a) All participants are reduired to complete an evaluation of coursesm,pctivities. 

□ b) Surveys are conducted through random selection of participdpt$C"'''' : ^ 

□ c) Feedback is encouraged, but provided at participants' discretion. \ 1 \ 

□ d) Feedback is obtained from staff meetings and peer discussions. % J / 

□ e) Other: , 1\ ^/ 

3 . Considering your district's conduct of evaluations, indicate by checking (✓) al that apply, 
how evaluations were structured: 

□ a) Through general or specific descriptions of training activity. ... 

□ b) Through questionnaires which contained muitipfe choice or scaled responses. 

□ c) Through interviews with trainees., 

□ d) Through verbal reports and other presentations by trainees to selected staff. 

4 . Considering your district's conduct of evaluations; specify by checking (✓) al that apply, to 
whom evaluations are submitted: v *> 

□ a) School Principal v.„ s 

□ b) District Chapter 2 Coordinator -^S v 

□ c) Chapter 2 Training Application Review Committee 

□ d) Local District trainer or Outside Consultant Providing Inservice Training 

□ e) Others > : ' ''_ , 

5 . Considering your district's conduct of evaluations, indicate by checking (✓) all that apply, he 
training and.prbfessibnal development evaluations are utilized in your district: 

□ a) For strategic-planning of future training and professional development activities. 

□ b) To provide feedback regarding specific training that was delivered. 

□ c) To reinforce primary emphasis of the professional development program. 

□ d) For breaking out future Chapter 2 staff training resources. 

6 . Considering your district's conduct of evaluations, specify by checking (✓) all that apply, to 
whom reports on training effectiveness and Impact are delivered: 

□ a) District Superintendent 

□ b) Departmental Chair 

□ b) Chapter 2 Coordinator 

□ c) Staff Developer | - 

□ d) Governing Board Members 



- Part 5 - 
Impressions of Arizona's Chapter 2 Program 

The statements below are Intended to solicit your impressions of the Chapter 2 Pjogranr > in 
general as well as provide feedback to the Arizona Department of Education (ADE) on me 
activities of the state Chapter 2 Office. Code your responses according to the .viewpoints 
suggested by indicating your level of agreement or disagreement with each statement. 

Circle the number which best describes your response based upon the range provided: 

,caj (A) (D) (SD) 

Strongly Agree Agree Disagree / JMP*M degree 

® ® ® /<?%, rf 



The Chapter 2 Program permits my district to offer a wid|,yarie1y, 
of training and professional development options. „ 

The monies allocated to the district permit purchdse of training 
and professional development activities that otherwise would 
not be available to district personnel. / \ v / 
Training and professional development,dbtivitie?,funded under 
Chapter 2 result in significant positiveyirripqcts for "qtjrisk*"^. 
students. \ .,. X 



LiA/sfe^lnteractions 



\i l 2 3 4 



Circle the number which hest describes your response based upon the range provided: 

(sa) , \ v ;w ,ny 

Strongly Agr6e - : Aflree 



>c*t (M (D) (SD) 

^\ ar ^ i Ai/ree Disagree Strongly Disagree 



® 

S A A D _SJL 



1 The appfteation packets and guidelines developed by ADE and 
used by your district to apply for Chapter 2 Program funding are 
conducive to Innovation In the design and implementation of 
educational projects and activities. 

2 . Chapter 2 staff provide appropriate technical assistance in the 
area of interpretation of program rules and regulations, 
allowability of proposed costs, and budgetary accuracy. 

3 Chapter 2 staff are competent In answering district questions 
regarding the application process and proper implementation of 
proposed educational projects and activities. 

4. Program monitoring, especially that which takes place at the 
time annua! applications are being certified and processed, is 
helpful In our district's operation of the Chapter 2 Program. 
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- Part 7 - 

Comments for Improvement of Chapter 2 Office Services 

Please indicate in the space below, any suggestions you feel would improve the type and lev 
of services the ADE Chapter 2 Office offers to districts. 

Please comment on staff performance, attitudes, technical knowledge, annual application 
process, federal regulations, accessibility of data and information, or any other matter you feel 
important. 




Thank you for your time and commitment. 



Descriptive Statistics 



Study Group (N«41) 

Alhambra Elementary 

Amphitheater Unified 

Apache Junction Unified 

Catalina Foothills Unified 

Chandler Unified 

Chino Valley Unified 

Crane Elementary 

Creighton Elementary 

Deer Valley Unified 

Flagstaff Unified 

Flowing Wells Unified 

Ganado Unified 

Gilbert Unified 

Glendale Elementary 

Glendale Union High 

Indian Oasis-Baboquivari Unified 

Isaac Elementary 

Kyrene Elementary 

Lake Havasu Unified 

Litchfield Elementary 

Madison Elementary 

Maricopa Unified 

Mesa Unified 

Page Unified 

Paradise Valley Unified 

Pendergast Elementary 

Peoria Unified 

Phoenix Elementary 

Phoenix Union High 

Roosevelt Elementary 

Scottsdale Unified 

Sierra Vista Unified 

Sunny side Unified 

Tempe Elementary 

Tempe Union High 

Tolleson Union High 

Tucson Unified 

Washington Elementary 

Williams Unified 

Yuma Elementary 

Yuma Union High 

Districts Not Responding (N=2) 
Laveen Elementary 
Marana Unified 
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Descriptive Statistics 
Parti. Respondent Demographics 

Ql.l Please indicate the number of years you have been responsible for coordinating training and 
professional development activities for you district? 

Min. Max. Mean Std. Dev. Total Responses 
0 10.0 3.8 2.4 41 

Ql .2 Is coordination of Chapter 2 projects and activities your primary job? 

Yes No 
Responses 3 38 

Percent 7% 93% 

Q1.3 Have you ever been a classroom teacher? 

Yes No 
Responses 41 0 
Percent 100% 0% 

If yes, how many total years of experience do you have as a classroom teacher? 

Min. Max. Mean Std. Dev. Total Responses 
3.0 21.0 10.0 4.9 41 
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Descriptive Statistics 

Part 2. Focus and Training Committee Review of Activities 

02 1 By checking all items that apply to your Chapter 2-supported program of Training & Pro- 
fessional Development, please indicate whether your district uses a committee structure tor 
purposes of: 

Responses 1 

Determining General Policies & Procedures 30 
Formulating Long Range Strategic Plans 31 
Reviewing Applications 24 
Deter minin g Types of Courses Offered 31 
Setting Limits on Funds Used 13 
Structured Committee Not Used 7 

Q2 2 If your district uses a committee structure to plan and/or approve expenditures for training & 
professional development, indicate all that apply to the membership represented on that 
committee. 

Responses 1 

Superintendent/Assistant Superintendent 26 
School Principal/Assistant Principal 27 
Governing Board Member 7 
Other District Administrators 22 
Certified Teacher 32 
Teacher Aide ® 
Other Instructional Staff 14 
Parents 23 
Private School Representatives 3 
Members of School/Business Partnerships 3 



How many individuals serve on this district Chapter 2 review committee? 

Min. Max. Mean Std. Dev. Total Responses 

4.0 30.0 13-9 6.4 41 



* Due to the possibility of multiple responses, total may not equal population count. 



Descriptive Statistics 



Part 2. Focus and Training Committee Review of Activities (continued) 

Q2.3 Consider the different methods by which Chapter 2 training activities are made avail- 
able to personnel in your district. Expressed as a proportion of the total Training & 
Professional Development Program, indicate the amount of emphasis your district 
places on the following areas (should total 100%): 



Q2.4 What would you say is the primary focus of Chapter 2-funded training and staff development 
activities in your district? 



Mini-Grants for Staff-Initiated Training 

District Mandated In-service 

Staff Requested In-services (Non-District) 



Min, 

0 
0 
0 



Max. 

50.0 

100.0 

80.0 



Mean 

20.4 
37.4 
41.8 



Std. Dev. 

15.0 
27.0 
24.5 



N 
41 
41 
41 



Responses 



Percent 



Improved Instructional Delivery 
More Effective Student Motivation 
Increased Staff Effectiveness 
Improved Curriculum 
School wide Improvements 
Better School Environment 
Increased Parental Involvement 
Unusable/No Response 



\A 

1 

15 
1 

2 
0 
0 
8 



42% 
3% 

45% 
3% 
6% 
0% 
0% 



TOTAL 



41 



100% 



ERIC 10 




CBaaip4<Bir S Cffl©ff(Mmai4<ii>ir ($rci<B0(t2<n>immmnir© 



Descriptive Statistics 
Part 3. District Process for Approving Staff Tr ainin g 

Q3.1 Are district personnel required to submit a formal application for Chapter 2-supported training 
and professional development activities? 

Yes No 
Responses 14 27 
Percent 4% 66% 

Q3.2 Is there a formal review process which must be addressed before training applications can be 
approved? 

Yes No 
Responses 14 25 
Percent 36% 64% 

Q3.3 Who makes the final decision to fund an individual request for attendance at training? 

Responses Percent 



5 13% 
9 23% 
1 3% 



District Superintendent 
Chapter 2 Coordinator 
Staff Developer 

Committee Members 4 10% 

Other 6 15% 

Principle 2 5% 

More Than One 12 31% 

Unusable/No Response 2 



'0 



TOTAL 



41 100% 



Q3.4 How many applications did you receive during the 1990-91 school year for participation in 
Chapter 2-funded staff training activities? 

Mim Max. Mean Std. Dev. N 

500.0 57.1 105.6 41 

Of the applications received, how many were not approved for funding? 

Min. Max. Mean Std. Dev. N 
80.0 5.7 16.7 41 

erJc 16: ii 
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Descriptive Statistics 



Part 3. District Process for Approving Staff Training (continued) 



Q3.5 Has your district established a formal limit on the dollar amount any one individual may 
expend in attendance at Chapter 2-funded training activities? 



Responses 
Percent 



Yes 

2 

5% 



No 

38 
95% 



Unusable/ 
No Response 1 



Dollar Limit Per Individual 
Dollar Limit Per Course 



Min. 

75.0 
75.0 



Max. 

250.0 
250.0 



Mean 
162.5 
162.5 



Std. Dev. 

123.7 
123.7 



Total 
41 
41 



Q3.6 Are there any district personnel who are required to participate in specific training and staff 
development activities? 

Yes No 
Responses 29 12 
Percent 71% 29% 



Indicate all categories that apply which best describe those individuals required to participate 
in some type of Chapter 2-funded training and professional development activity during the 
1990-91 school year: 





Responses 1 


New Principals/Assistant Principals 


16 


Current Principals/Assistant Principals 


11 


New Board Members 


0 


Current Board Members 


0 


Other New/Current Administrators 


6 


New Certified Teachers 


26 


Current Certified Teachers 


12 


New Teacher Aides 


7 


Current Teacher Aides 


5 


Other New/Current Instructional Staff 


7 


New librarians 


8 


Current Librarians 


5 


New Counselors 


10 


Current Counselors 


7 


Others 


7 



12 



1 Due to the possibility of multiple responses, total may not equal population count. 
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Descriptive Statistics 

Part 3. District Process for Approving Staff Training (continued) 

Q3.7 Are there any subject areas that require instructional personnel to receive training in new 
content or innovative instructional methods on an annual basis? 

Yes No 
Responses 41 0 
Percent 100% 0% 
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Descriptive Statistics 



Part 4. District Training Evaluation & Planning Processes 



Q4.1 Does your district formally evaluate participation in T&PD activities? 



Responses 
Percent 



Yes 
27 
66% 



No 
14 

34% 



Q4.2 Considering you districts conduct of evaluations, indicate the primary method by which staff 
training evaluations are obtained: 



All Complete Evaluation 
Random Sample Survey 
Feedback Via Staff 
Meetings / Peer Discussion 
Other 



Responses 
23 
1 

5 
12 



Percent 

56% 
2% 

12% 
29% 



TOTAL 



41 



100% 



Q4.3 Considering your districts conduct of evaluations, indicate how evaluations were structured: 



Other Methods 
Descriptions of Training 
Questionnaires 
Interviews 



Responses 1 

0 
23 
25 
12 



Q4.4 Considering your districts conduct of evaluations, indicate to whom evaluations are submitted: 



School Principal 

Chapter 2 Coordinator 

Chapter 2 Application Review Committee 

Local District Trainer/Outside Consultant 

Other 



Responses 1 
15 
27 

8 

19 

11 



9 

ERIC 
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Descriptive Statistics 

Part 4. District Training Evaluation & Planning Processes (continued) 

Q4.5 Considering your districts conduct of evaluations, indicate how training and professional 
development evaluations are utilized: 

Responses 1 

Planning for Future Training 28 
Provide Feedback 

Reinforce Primary Emphasis ^ 
Breaking Out Future Resources 

Q4.6 Considering your districts conduct of evaluations, specify to whom reports on training 
effectiveness and impact are delivered: 

Responses 1 

25 

District Superintendent " 
Departmental Chair 

Chapter 2 Coordinator 6 
Staff Developer 



Governing Board Members 



15 



l r^e to the possibility of multiple responses, total may not equal population count. 

ERIC Lbu is 



Descriptive Statistics 
PartS. Impressions of Arizona Chapter 2 Program 



Q5.1 Chapter 2 permits my district to offer a wide variety of training and professional development 
options. 

Strongly Agree Disagree Strongly 
Agree Disagree 
Responses 31 9 1 0 

Percent 76% 22% 2% 0% 



Q5.2 The monies allocated to the district permit purchase of training and professional development 
activities that otherwise would not be available to district personnel. 

Strongly Agree Disagree Strongly 

Agree Disagree 

Responses 31 9 1 0 

Percent 76% 22% 2% 0% 



Q5.3 Training and professional development activities funded under Chapter 2 result in significant 
positive impacts for at-risk students. 

Strongly Agree Disagree Strongly 

Agree Disagree 
Responses 22 18 1 0 

Percent 54% 44% 2% 0% 
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Descriptive Statistics 
Part 6. LEA/SEA Interactions 

06 1 The application packets and guidelines developed by ADE and used by your district to apply for 
Chapter 2 Program funding are conducive to innovation in the design and implementation of 
educational projects and activities. 

Strongly Agree Disagree Strongly 

Agree Disagree 

Responses 18 23 0 0 

Percent 44% 56% 0% 0% 

Q6.2 Chapter 2 staff provide appropriate technical assistance in the area of interpretation of pro- 
gram rules and regulations, allowability of proposed costs, r<*>d budgetary accuracy. 

Strongly Agree Disagree Strongly 

Agree Disagree 

Responses 21 20 0 0 

Percent 51% 49% 0% 0% 



06 3 Chapter 2 staff are competent in answering district questions regarding the application process 
and proper implementation of proposed educational projects and activities. 

Strongly Agree Disagree Strongly 

Agree Disagree 

Responses 27 14 0 0 

Percent 66% 34% 0% 0% 



06 4 Program monitoring, especially that which takes place at the time annual applications are 
being certified and processed, is helpful in our district's operation of the Chapter 2 Program. 

Strongly Agree Disagree Strongly 

Agree Disagree 

Responses 18 21 2 0 

Percent 44% 51% 5% 0% 
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Summary of Statistical Findings 

Part 1. Respondent Demographics 



• Average number of years responsible for coordinating training and professional 
development activities (T&PD): 

Average Number of Years: 4 
Longest Number of Years : 10 

• 95 percent of the respondents indicated that coordination of Chapter 2 projects is NOT their 
primary job responsibility. 

• All of the respondents indicated that they had been classroom teachers* 

Average years as classroom teacher: 10 
Longest years as classroom teacher: 21 
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Summary op Statistical Findings 



Part 2. Focus and Training Committee Review of Activities 

• Use of Committee structure: 

Over 75 percent of the districts indicated that committee structures are used to 

- determine general policies and procedures 

- formulate long-range strategic plans 

- determine the types of courses offered to staff 

60 percent use committee structure to review applications for T&PD 

35 percent use committee structure to set limits on the use of funds 

13 percent DO NOT use committee structures in Chapter 2 T&PD activities 

• Committee Membership: High 82% -Teachers 

66% - Superintendents 
68% - Principals 
, 58% - Other District Personnel 

-y 58% - Parents 

Low 34% - Other Instructional Staff 
8% - Private Schools 

5% - Members of School/District Partnerships 

• Average size of Chapter 2 Committees: 13.63 members 

Note: One district reported 60 members on the committee overseeing Chapter 2 activities. 

• Emphasis of Chapter 2 Activities: 

Number of districts indicating primary area of emphasis (excludes districts reporting equal 
weights among selections): Valid N = 31 

Mini-Grants: 0 

District-Mandated In-service: 9 

District-Sponsored In-service: 9 

Staff-Requested In-service: 13 

Other: 0 

Note: 18 out of 31 districts, or 58 percent, report district-supported in-service activities as the 
primary area of emphasis* 

• Primary Focus of Chapter 2-funded T&PD Activities: 

87 percent of the districts reporting identified either Improved Instructional Delivery (39%) or 
Increase Staff Effectiveness (48%) as the primary focus of Chapter 2-funded T&PD activities. 
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Summary of Statistical Findings 

Part & District Process for Approving Staff Training 

• 63 percent of all reporting districts DO NOT require a formal application to participate in 
Chapter 2-funded T&PD activities. 

• 62 percent indicate no formal process for reviewing applications for T&PD activities. 

• Who makes the final decision to fund a request to participate in T&PD activities (allowing for 
multiple responses): 

34% indicated the Chapter 2 Coordinator (13 districts) 
21% indicated District Administration (8 districts) 
21% indicated Staff Developer (8 districts) 

• 83 percent of the districts responding indicated that they received a maximum of 60 requests 
for participation in T&PD activities during the 1990-91 school year. 

The average T&PD rejection rate for these districts was 2.36. Two districts reported rejection 
rates in excess of 80 percent, one at 33 percent. 

• 95 percent of the districts responding indicate no formal upper limit to the amount of funds any 
individual may receive to participate in T&PD activities. 

• Required participation in T&PD activities: 

71% of the districts indicated that certain staff persons are required to attend T&PD activities. 
Of those required: 

89% are New Teachers 

59% are New Principals or Assistant Principals 

• Required Annual Training: 

11 districts indicated that they require annual training for instructional personnel in specific 
content areas. The content areas cited are: 

Thinking Skills 

Essential Elements of Instruction (3 districts) 
Science 

Adaptive Curriculum 
Language Arts 
Chemical Abuse Prevention 
Effective Schools 
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Summary of Statistical Findings 

Part 4. District Training and Evaluation and Planning Processes 

• 63 percent indicate they conduct formal evaluations of T&PD activities. 

• 66 percent indicate that all participants complete evaluations 

39 percent indicate that evaluations are completed through staff meetings and peer 
discussions. 

Note: The percentages cited above allow for multiple responses. 

• Individual to Whom T&PD evaluations are submitted: 

79 percent indicated District Chapter 2 Coordinator or District Administration 
50 percent indicated the providers of the in-service training 

• 95 percent of the districts reported that the T&PD evaluations are used to provide feedback on 
the specific training that was delivered. 

66 percent reported T&PD evaluations are utilized for strategic planning of future T&PD 
activities. 

• 79 percent indicated that training effectiveness reports are delivered to the Staff 
Development Coordinator. 71 percent indicated delivery to the Chapter 2 Coordinator 
and 61 percent indicated the District Superintendent as a recipient. 
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Summary of Statistical Findings 

Part 5. Impressions of Arizona's Chapter 2 Program 



Domain: Statements focus on the usefulness of Chapter 2 programs and monies to permit 
offering a wider variety of T&PD activities that otherwise would not be possible. 

Survey Questions: Part 5.1, 5.2 

Strongly 

Agree/Disagree Agree/Disagree Combined 
Positive 24% 74% 98% 

Negative 2% 0% 2% 



Domain: Statement on the positive impact that T&PD activities have on at-risk students . 
Survey Question: Part 5.3 

Strongly 

Agree/Disagree Agree/Disagree Combined 
Positive 45% 52% 97% 

Negative 3% 0% 3% 
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Summary of Statistical Findings 

Part 6. LEA/SEA Interactions 



Domain: Statement on usefulness of ADE application and guideline materials to promote 
innovation in the design and implementation of educational projects and 
activities. 

Survey Questions: Part 5.1 

Strongly 

Agree/Disagree Agree/Disagree Combined 
Positive 58% 42% 100% 

Negative 0% 0% 0% 



Domain: Statements on the usefulness of the assistance received from the ADE Chapter 
2 Office. 

Survey Question: Part 5.2, 5.3 

Strongly 

Agree/Disagree Agree/Disagree Combined 
Positive 43% 57% 100% 

Negative 0% 0% 0% 



Domain: Statement on the usefulness of ADE program monitoring to help districts with 
their operation of Chapter 2 T&PD activities. 

Survey Question: Part 5.4 

Strongly 

Agree/Disagree Agree/Disagree Combined 
Positive 53% 42% 95% 

Negative 5% 0% 5% 
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